Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
1.875291549s
FAIL (2025-06-30 16:44:32 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.87531214s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
2.037779108s
FAIL (2025-06-30 16:43:31 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 2.037808264s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
2.023772644s
FAIL (2025-06-30 16:42:32 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 2.023805537s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
1.876487617s
FAIL (2025-06-30 16:41:48 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.876511984s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
2.095994297s
FAIL (2025-06-30 16:41:12 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 2.096021228s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
1.834254157s
FAIL (2025-06-30 16:40:56 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.834282181s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
1.954587863s
FAIL (2025-06-30 16:40:38 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.954614635s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
1.83850277s
FAIL (2025-06-30 16:40:09 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.838532728s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
1.862257324s
FAIL (2025-06-30 16:39:11 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.862283513s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
1.91035474s
FAIL (2025-06-30 16:38:25 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.910379939s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
1.840367564s
FAIL (2025-06-30 16:37:11 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.840393563s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
1.839019695s
FAIL (2025-06-30 16:36:15 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.839046756s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
1.864247671s
FAIL (2025-06-30 16:35:08 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.864272408s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
1.876304287s
FAIL (2025-06-30 16:34:09 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.876332411s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
1.844411041s
FAIL (2025-06-30 16:33:47 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.844435868s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
1.866317451s
FAIL (2025-06-30 16:33:24 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.866349914s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
1.88354022s
FAIL (2025-06-30 16:32:32 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.883567021s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
1.870866373s
FAIL (2025-06-30 16:31:50 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.870918954s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
1.859032684s
FAIL (2025-06-30 16:31:31 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.859071508s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
1.888573819s
FAIL (2025-06-30 16:31:15 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.888599639s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
1.868710889s
FAIL (2025-06-30 16:30:25 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.868736137s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
1.873345736s
FAIL (2025-06-30 16:29:32 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.873373178s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
1.895159448s
FAIL (2025-06-30 16:29:08 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.89518675s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
1.941180443s
FAIL (2025-06-30 16:28:35 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.941210711s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
1.900270868s
FAIL (2025-06-30 16:28:08 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.900298218s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
1.858516373s
FAIL (2025-06-30 16:27:09 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.858535961s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
1.926520865s
FAIL (2025-06-30 16:26:08 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.926538789s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
1.868939995s
FAIL (2025-06-30 16:25:13 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.86897317s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
1.959048776s
FAIL (2025-06-30 16:24:15 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.95907756s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
1.911770977s
FAIL (2025-06-30 16:23:09 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.911939008s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
1.864506735s
FAIL (2025-06-30 16:21:58 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.864532255s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
1.843555159s
FAIL (2025-06-30 16:21:35 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.843581429s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
1.84592205s
FAIL (2025-06-30 16:21:01 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.845956297s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
1.889070249s
FAIL (2025-06-30 16:20:18 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.889094103s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
1.90694708s
FAIL (2025-06-30 16:19:45 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.906976638s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
1.987989255s
FAIL (2025-06-30 16:18:49 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.988036155s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
1.89721478s
FAIL (2025-06-30 16:17:37 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.897239818s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
2.0000087s
FAIL (2025-06-30 16:16:49 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 2.000036122s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
1.93149841s
FAIL (2025-06-30 16:15:35 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.931531454s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
2.294245666s
FAIL (2025-06-30 16:14:51 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 2.294275092s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
1.945297069s
FAIL (2025-06-30 16:13:53 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.945322437s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
1.962632975s
FAIL (2025-06-30 16:13:15 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.962684043s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
1.917724617s
FAIL (2025-06-30 16:12:07 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.917748343s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
1.893086729s
FAIL (2025-06-30 16:11:46 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.89311237s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
1.893988535s
FAIL (2025-06-30 16:10:54 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.894015937s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
1.85633499s
FAIL (2025-06-30 16:09:50 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.856367942s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
1.858968505s
FAIL (2025-06-30 16:09:28 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.859004073s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
1.901046217s
FAIL (2025-06-30 16:08:22 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.90106799s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
1.8724975s
FAIL (2025-06-30 16:07:38 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.872526427s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
1.864548594s
FAIL (2025-06-30 16:06:24 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 524636s) [upload=139638]: Starting upload Jun 30 15:50:30.567977240 (174h40m5.338618575s) [upload=139638]: Done starting upload Jun 30 15:50:30.568059187 (174h40m5.338700522s) [upload=139638]: Uploading segments... Jun 30 15:50:30.568118791 (174h40m5.338760126s) [upload=139638]: Got next segment. Inline: true Jun 30 15:50:30.568124332 (174h40m5.338765667s) [upload=139638]: Waiting for error group managing segments... Jun 30 15:50:30.568164378 (174h40m5.338805714s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:50:30.568169348 (174h40m5.338810683s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:50:30.568171883 (174h40m5.338813218s) [upload=139638]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:50:30.866704396 (174h40m5.637345721s) [upload=139638]: Done uploading segments... Jun 30 15:50:30.866722359 (174h40m5.637363694s) [upload=139638]: Upload finished. err: <nil> Jun 30 15:51:29.670570472 (174h41m4.441211807s) [upload=139654]: Starting upload Jun 30 15:51:29.670657829 (174h41m4.441299164s) [upload=139654]: Done starting upload Jun 30 15:51:29.670713255 (174h41m4.44135459s) [upload=139654]: Uploading segments... Jun 30 15:51:29.670751728 (174h41m4.441393073s) [upload=139654]: Got next segment. Inline: true Jun 30 15:51:29.670757058 (174h41m4.441398403s) [upload=139654]: Waiting for error group managing segments... Jun 30 15:51:29.670799430 (174h41m4.441440765s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:51:29.670805571 (174h41m4.441446907s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:51:29.670808527 (174h41m4.441449862s) [upload=139654]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:51:29.937520995 (174h41m4.70816233s) [upload=139654]: Done uploading segments... Jun 30 15:51:29.937541113 (174h41m4.708182458s) [upload=139654]: Upload finished. err: <nil> Jun 30 15:52:32.854941993 (174h42m7.625583328s) [upload=139667]: Starting upload Jun 30 15:52:32.855093323 (174h42m7.625734658s) [upload=139667]: Done starting upload Jun 30 15:52:32.855269430 (174h42m7.625910775s) [upload=139667]: Uploading segments... Jun 30 15:52:32.855329234 (174h42m7.625970579s) [upload=139667]: Got next segment. Inline: true Jun 30 15:52:32.855334484 (174h42m7.625975819s) [upload=139667]: Waiting for error group managing segments... Jun 30 15:52:32.855370704 (174h42m7.626012039s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:52:32.855375443 (174h42m7.626016778s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:52:32.855377957 (174h42m7.626019292s) [upload=139667]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:52:33.134763960 (174h42m7.905405295s) [upload=139667]: Done uploading segments... Jun 30 15:52:33.134782094 (174h42m7.905423429s) [upload=139667]: Upload finished. err: <nil> Jun 30 15:53:34.942739271 (174h43m9.713380606s) [upload=139680]: Starting upload Jun 30 15:53:34.942788094 (174h43m9.71342943s) [upload=139680]: Done starting upload Jun 30 15:53:34.942833861 (174h43m9.713475196s) [upload=139680]: Uploading segments... Jun 30 15:53:34.942860723 (174h43m9.713502058s) [upload=139680]: Got next segment. Inline: true Jun 30 15:53:34.942864500 (174h43m9.713505825s) [upload=139680]: Waiting for error group managing segments... Jun 30 15:53:34.942896372 (174h43m9.713537707s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:53:34.942901040 (174h43m9.713542375s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:53:34.942903555 (174h43m9.71354488s) [upload=139680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:53:35.216328894 (174h43m9.986970229s) [upload=139680]: Done uploading segments... Jun 30 15:53:35.216347370 (174h43m9.986988705s) [upload=139680]: Upload finished. err: <nil> Jun 30 15:54:27.100838275 (174h44m1.87147963s) [upload=139692]: Starting upload Jun 30 15:54:27.100954046 (174h44m1.871595381s) [upload=139692]: Done starting upload Jun 30 15:54:27.101028509 (174h44m1.871669844s) [upload=139692]: Uploading segments... Jun 30 15:54:27.101090678 (174h44m1.871732063s) [upload=139692]: Got next segment. Inline: true Jun 30 15:54:27.101097190 (174h44m1.871738555s) [upload=139692]: Waiting for error group managing segments... Jun 30 15:54:27.101141224 (174h44m1.87178256s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:54:27.101146695 (174h44m1.87178803s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:54:27.101148939 (174h44m1.871790274s) [upload=139692]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:54:27.561979059 (174h44m2.332620394s) [upload=139692]: Done uploading segments... Jun 30 15:54:27.561995720 (174h44m2.332637045s) [upload=139692]: Upload finished. err: <nil> Jun 30 15:55:05.413255672 (174h44m40.183897007s) [upload=139699]: Starting upload Jun 30 15:55:05.413307431 (174h44m40.183948766s) [upload=139699]: Done starting upload Jun 30 15:55:05.413367466 (174h44m40.184008801s) [upload=139699]: Uploading segments... Jun 30 15:55:05.413428423 (174h44m40.184069758s) [upload=139699]: Got next segment. Inline: true Jun 30 15:55:05.413433553 (174h44m40.184074878s) [upload=139699]: Waiting for error group managing segments... Jun 30 15:55:05.413473880 (174h44m40.184115215s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:05.413480332 (174h44m40.184121667s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:05.413483138 (174h44m40.184124473s) [upload=139699]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:05.688544537 (174h44m40.459185863s) [upload=139699]: Done uploading segments... Jun 30 15:55:05.688563043 (174h44m40.459204378s) [upload=139699]: Upload finished. err: <nil> Jun 30 15:55:56.133377923 (174h45m30.904019258s) [upload=139710]: Starting upload Jun 30 15:55:56.133437206 (174h45m30.904078541s) [upload=139710]: Done starting upload Jun 30 15:55:56.133491981 (174h45m30.904133306s) [upload=139710]: Uploading segments... Jun 30 15:55:56.133525956 (174h45m30.904167291s) [upload=139710]: Got next segment. Inline: true Jun 30 15:55:56.133531045 (174h45m30.90417238s) [upload=139710]: Waiting for error group managing segments... Jun 30 15:55:56.133566113 (174h45m30.904207448s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:55:56.133570922 (174h45m30.904212257s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:55:56.133573517 (174h45m30.904214852s) [upload=139710]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:55:56.414698989 (174h45m31.185340314s) [upload=139710]: Done uploading segments... Jun 30 15:55:56.414719389 (174h45m31.185360724s) [upload=139710]: Upload finished. err: <nil> Jun 30 15:56:11.431261733 (174h45m46.201903068s) [upload=139714]: Starting upload Jun 30 15:56:11.431311408 (174h45m46.201952743s) [upload=139714]: Done starting upload Jun 30 15:56:11.431366413 (174h45m46.202007738s) [upload=139714]: Uploading segments... Jun 30 15:56:11.431395789 (174h45m46.202037125s) [upload=139714]: Got next segment. Inline: true Jun 30 15:56:11.431399346 (174h45m46.202040681s) [upload=139714]: Waiting for error group managing segments... Jun 30 15:56:11.431430856 (174h45m46.202072191s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:56:11.431435445 (174h45m46.20207678s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:56:11.431438210 (174h45m46.202079545s) [upload=139714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:56:11.714407860 (174h45m46.485049195s) [upload=139714]: Done uploading segments... Jun 30 15:56:11.714426035 (174h45m46.48506736s) [upload=139714]: Upload finished. err: <nil> Jun 30 15:57:23.326036780 (174h46m58.096678115s) [upload=139729]: Starting upload Jun 30 15:57:23.326120901 (174h46m58.096762236s) [upload=139729]: Done starting upload Jun 30 15:57:23.326178000 (174h46m58.096819335s) [upload=139729]: Uploading segments... Jun 30 15:57:23.326210263 (174h46m58.096851598s) [upload=139729]: Got next segment. Inline: true Jun 30 15:57:23.326213970 (174h46m58.096855305s) [upload=139729]: Waiting for error group managing segments... Jun 30 15:57:23.326247654 (174h46m58.096888979s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:57:23.326252303 (174h46m58.096893628s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:57:23.326290185 (174h46m58.09693152s) [upload=139729]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:57:23.603822151 (174h46m58.374463516s) [upload=139729]: Done uploading segments... Jun 30 15:57:23.603841207 (174h46m58.374482542s) [upload=139729]: Upload finished. err: <nil> Jun 30 15:58:37.844911786 (174h48m12.615553121s) [upload=139743]: Starting upload Jun 30 15:58:37.844975849 (174h48m12.615617184s) [upload=139743]: Done starting upload Jun 30 15:58:37.845024942 (174h48m12.615666277s) [upload=139743]: Uploading segments... Jun 30 15:58:37.845052084 (174h48m12.615693419s) [upload=139743]: Got next segment. Inline: true Jun 30 15:58:37.845055701 (174h48m12.615697026s) [upload=139743]: Waiting for error group managing segments... Jun 30 15:58:37.845086240 (174h48m12.615727575s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 30 15:58:37.845090979 (174h48m12.615732314s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 30 15:58:37.845093303 (174h48m12.615734638s) [upload=139743]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 30 15:58:38.118447969 (174h48m12.889089294s) [upload=139743]: Done uploading segments... Jun 30 15:58:38.118466284 (174h48m12.889107619s) [upload=139743]: Upload finished. err: <nil>
Duration: 1.864575886s
core
uplink
us-select-1
|