Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
1.889045554s
FAIL (2025-07-04 12:24:31 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.889073527s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
1.887934175s
FAIL (2025-07-04 12:24:02 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.887960024s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
1.857637315s
FAIL (2025-07-04 12:22:49 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.85767714s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
1.96822032s
FAIL (2025-07-04 12:22:27 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.96825129s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
1.858602291s
FAIL (2025-07-04 12:21:29 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.858629725s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
1.865155729s
FAIL (2025-07-04 12:20:56 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.865190485s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
1.891824669s
FAIL (2025-07-04 12:20:02 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.891855157s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
1.922960065s
FAIL (2025-07-04 12:18:59 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.922979743s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
1.921680384s
FAIL (2025-07-04 12:18:13 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.921707256s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.46349063s
FAIL (2025-07-04 12:17:23 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 2.463512682s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
1.918779358s
FAIL (2025-07-04 12:16:41 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.918805637s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
1.930222065s
FAIL (2025-07-04 12:15:37 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.930259578s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
1.901497437s
FAIL (2025-07-04 12:15:20 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.901533987s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
1.871351345s
FAIL (2025-07-04 12:14:55 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.871375943s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
1.898761068s
FAIL (2025-07-04 12:14:09 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.898786096s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.858044835s
FAIL (2025-07-04 12:13:36 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.858077928s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
1.90060033s
FAIL (2025-07-04 12:13:11 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.900622011s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
1.883322283s
FAIL (2025-07-04 12:12:45 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.883345145s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
1.908493766s
FAIL (2025-07-04 12:12:09 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.908520709s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
2.079315284s
FAIL (2025-07-04 12:11:12 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 2.079346654s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
1.88458886s
FAIL (2025-07-04 12:10:04 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.884631661s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
1.867659211s
FAIL (2025-07-04 12:09:43 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.867683047s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
1.878483157s
FAIL (2025-07-04 12:08:54 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.878522371s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.000276402s
FAIL (2025-07-04 12:08:52 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 30.000297344s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
5.686907019s
FAIL (2025-07-04 10:26:24 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.68693936s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
5.646163793s
FAIL (2025-07-04 10:25:48 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.646204489s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
5.711062244s
FAIL (2025-07-04 10:25:15 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.71108636s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
5.705919377s
FAIL (2025-07-04 10:24:33 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.705944887s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
5.717870117s
FAIL (2025-07-04 10:23:54 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.717894423s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
5.653496067s
FAIL (2025-07-04 10:22:56 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.653521086s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
5.698753235s
FAIL (2025-07-04 10:22:09 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.698774656s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
5.701687566s
FAIL (2025-07-04 10:21:41 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.701712133s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
5.643113302s
FAIL (2025-07-04 10:20:35 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.643143058s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
5.685203563s
FAIL (2025-07-04 10:19:42 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.68523848s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
5.752052282s
FAIL (2025-07-04 10:18:59 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.752075937s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
5.703410143s
FAIL (2025-07-04 10:18:36 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.703436775s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
5.630014114s
FAIL (2025-07-04 10:18:07 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.630038692s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
5.950473469s
FAIL (2025-07-04 10:17:52 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.950503147s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
5.689220608s
FAIL (2025-07-04 10:16:37 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.689245204s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
5.658325853s
FAIL (2025-07-04 10:16:08 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.658346783s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
5.685066075s
FAIL (2025-07-04 10:15:12 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.685111703s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.686662074s
FAIL (2025-07-04 10:14:40 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.686690549s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
5.677041263s
FAIL (2025-07-04 10:13:36 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.677075869s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
5.686735552s
FAIL (2025-07-04 10:13:18 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.68675532s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
5.665640654s
FAIL (2025-07-04 10:12:30 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.665672877s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
6.417627146s
FAIL (2025-07-04 10:11:18 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 6.417652054s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
6.428869174s
FAIL (2025-07-04 10:10:33 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 6.428897659s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
5.727548023s
FAIL (2025-07-04 10:10:06 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.72757241s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
5.717585904s
FAIL (2025-07-04 10:09:18 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 5.717613747s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
10.887962376s
FAIL (2025-07-04 10:08:43 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
9:44:49.532338244 (264h34m24.302979579s) [noise=true , address=45.91.116.13:4154 , piece_id=ZXIDO4ZDGP5MKRLPHIO63FFMT2GYB3FA55AYQTRCBKZWAVKQU2OA, seg_pos={0 0}, upload=211553]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 4 09:44:49.532380785 (264h34m24.30302212s) [seg_pos={0 0}, upload=211553]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:44:49.532463513 (264h34m24.303104848s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:44:49.532468273 (264h34m24.303109598s) [upload=211553]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:44:49.835200025 (264h34m24.60584136s) [upload=211553]: Done uploading segments...
Jul 4 09:44:49.835220213 (264h34m24.605861538s) [upload=211553]: Upload finished. err: <nil>
Jul 4 09:45:20.888561003 (264h34m55.659202328s) [upload=211562]: Starting upload
Jul 4 09:45:20.888619606 (264h34m55.659260941s) [upload=211562]: Done starting upload
Jul 4 09:45:20.888709688 (264h34m55.659351023s) [upload=211562]: Uploading segments...
Jul 4 09:45:20.888790152 (264h34m55.659431487s) [upload=211562]: Got next segment. Inline: false
Jul 4 09:45:20.888818907 (264h34m55.659460232s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 4 09:45:20.888823406 (264h34m55.659464741s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 4 09:45:21.120358782 (264h34m55.891000117s) [seg_pos={0 0}, upload=211562]: Begin upload segment...
Jul 4 09:45:21.120408657 (264h34m55.891049993s) [seg_pos={0 0}, upload=211562]: Done begin upload segment.
Jul 4 09:45:21.120432012 (264h34m55.891073347s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.120497517 (264h34m55.891138852s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121293831 (264h34m55.891935166s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121377481 (264h34m55.892018816s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121879170 (264h34m55.892520506s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.121948212 (264h34m55.892589557s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122256483 (264h34m55.892897818s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122582075 (264h34m55.89322341s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.122799321 (264h34m55.893440656s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.123662221 (264h34m55.894303566s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.124798645 (264h34m55.89543998s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125161430 (264h34m55.895802765s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125490799 (264h34m55.896132134s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.125860698 (264h34m55.896502033s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126173075 (264h34m55.89681441s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:21.126474482 (264h34m55.897115817s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Uploading piece...
Jul 4 09:45:22.819727996 (264h34m57.590369321s) [upload=211562]: Next returned nil segment
Jul 4 09:45:22.819748425 (264h34m57.59038979s) [upload=211562]: Waiting for error group managing segments...
Jul 4 09:45:23.107090303 (264h34m57.877731638s) [noise=true , address=104.36.82.8:4970 , piece_id=LESLTOIQ4ZOQUUYATDL6OHAJAJPKFY55YZAWNVEFTOALYFVQEV6Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.109255576 (264h34m57.879896911s) [noise=true , address=104.36.82.2:4730 , piece_id=3JDG7XWYVEMVCYAM444JRZLMGIPHEOZYPILESW6NGFKELZX5JE3A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.150369180 (264h34m57.921010515s) [noise=true , address=104.36.82.16:4295 , piece_id=GYJPYHPLAUXPJF72CAV4F6ACXNVF3OY63SODYRVP3MWGCXUOVBYQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.172864695 (264h34m57.94350603s) [noise=true , address=104.36.82.5:4840 , piece_id=WNOAEUFV7KU45GK7QONOZZUEIUHABLTVO73HYFAEC2KIGG5I7ZXA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.182147795 (264h34m57.95278913s) [noise=true , address=104.36.82.2:4731 , piece_id=6G2OOJVZQXXIBSYY2NORBL4TJFOQFNWGTX5J65OHY4YNZF43AOVA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.190925617 (264h34m57.961566952s) [noise=true , address=45.91.116.11:4118 , piece_id=W5FTWPFIVDHXUIQOA2HB42Q2O6IG3QZKL36ZLP4F3KRV6YUOXYQQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.208781678 (264h34m57.979423013s) [noise=true , address=45.91.116.6:4973 , piece_id=FDI3EWGW5JDK4QMLT3YPRQEEEJEJPJPBV6GZ46R4FQQ3PLFOGSMQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.220373053 (264h34m57.991014388s) [noise=true , address=77.90.31.14:4661 , piece_id=UP2X2RVD65C4TXDN3DQRGP6DQK34C5IRBZUR4OQP3DJQPSWXFFFQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.222591687 (264h34m57.993233023s) [noise=true , address=77.90.31.12:4592 , piece_id=KQAJZ7F6PWNLDYKFPS4KGRRCGBTGFSU5IA3M6MQTDIIJEBPGFS6A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.223558928 (264h34m57.994200263s) [noise=true , address=45.91.116.5:4297 , piece_id=ZMHMXLTE5NQIJNH4COYSKZ2PRC6QK4WIGTZMA3RWJKWF6KJQR7WA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.226570169 (264h34m57.997211504s) [noise=true , address=45.91.116.12:4226 , piece_id=YSA5JIWGZ6GNBXQI4EUNPG2UJJECHMYEGZ5WYMKQYYIEZE2BGAKA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229085010 (264h34m57.999726345s) [noise=true , address=45.91.116.13:4155 , piece_id=WXH755TXLQSSIE72TSHB6HEDPO334ORRCMNVZGRYJIA6TTEJ733Q, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229280273 (264h34m57.999921609s) [noise=true , address=45.91.116.6:4974 , piece_id=VTGANFQFLH5BZAPX2EPQ54DCPM3KCUXSL4QX57YZM6PRGXDWI4VA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: <nil>
Jul 4 09:45:23.229314208 (264h34m57.999955543s) [seg_pos={0 0}, upload=211562]: Segment reached optimal threshold of 13 pieces.
Jul 4 09:45:23.229471660 (264h34m58.000112995s) [noise=true , address=77.90.31.14:4663 , piece_id=R7QXMGSE5XIBRVYUAJ2NFV7HNCRE5PAYJ5MGTL6O6KBLP5555ZWA, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229494743 (264h34m58.000136078s) [noise=true , address=77.90.31.11:4553 , piece_id=ZSP3BX5DTCHAB4AFLKYPQMOX6ECIUNQSUNPQORSLM7SZKMGG4UBQ, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229557454 (264h34m58.000198779s) [noise=true , address=77.90.31.13:4626 , piece_id=HNONPFWDNKXWOLSHUUSUL7XKEDEAPVXXXOTJV3O6THBZEIW76N4A, seg_pos={0 0}, upload=211562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
Jul 4 09:45:23.229573915 (264h34m58.00021524s) [seg_pos={0 0}, upload=211562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 4 09:45:23.229655020 (264h34m58.000296355s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 4 09:45:23.229659348 (264h34m58.000300683s) [upload=211562]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 4 09:45:23.520080448 (264h34m58.290721783s) [upload=211562]: Done uploading segments...
Jul 4 09:45:23.520102610 (264h34m58.290743945s) [upload=211562]: Upload finished. err: <nil>
Duration: 10.887993406s
core
uplink
us-select-1
|