Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.290611019s
FAIL (2025-07-04 21:41:13 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:51362->45.38.24.2:443: read: connection reset by peer
Duration: 2.290642209s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.309662244s
FAIL (2025-07-04 21:40:01 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:51362->45.38.24.2:443: read: connection reset by peer
Duration: 2.309683587s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
485.109499ms
FAIL (2025-07-04 21:38:52 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:51362->45.38.24.2:443: read: connection reset by peer
Duration: 485.14141ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
1.894456452s
FAIL (2025-07-04 12:50:33 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.894487222s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
1.877684958s
FAIL (2025-07-04 12:49:35 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.8777072s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
1.899524832s
FAIL (2025-07-04 12:49:06 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.899550411s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
1.967224898s
FAIL (2025-07-04 12:48:43 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.967245166s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
1.889266516s
FAIL (2025-07-04 12:47:38 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.889291814s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
1.8690449s
FAIL (2025-07-04 12:47:05 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.86907086s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
1.88711491s
FAIL (2025-07-04 12:46:50 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.887140918s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
1.874562065s
FAIL (2025-07-04 12:46:18 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.874578856s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
1.889867375s
FAIL (2025-07-04 12:45:35 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.889900017s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
1.929320664s
FAIL (2025-07-04 12:44:38 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.929345621s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
1.883785218s
FAIL (2025-07-04 12:44:04 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.883804645s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
1.929648698s
FAIL (2025-07-04 12:43:39 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.929683665s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
1.929637486s
FAIL (2025-07-04 12:43:14 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.92967134s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
1.875044346s
FAIL (2025-07-04 12:42:10 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.875097007s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
2.053973914s
FAIL (2025-07-04 12:41:32 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 2.05399273s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
1.933573479s
FAIL (2025-07-04 12:40:29 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.933599909s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
1.899811459s
FAIL (2025-07-04 12:39:27 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.899846097s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
1.890632636s
FAIL (2025-07-04 12:38:25 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.890656703s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
1.906070821s
FAIL (2025-07-04 12:37:54 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.906094788s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
2.105010572s
FAIL (2025-07-04 12:37:37 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 2.10504041s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
1.852065226s
FAIL (2025-07-04 12:36:50 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.852091816s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
1.905069581s
FAIL (2025-07-04 12:35:51 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.905093526s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
1.898432898s
FAIL (2025-07-04 12:35:10 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.898450532s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
2.053903399s
FAIL (2025-07-04 12:34:52 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 2.053932274s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
2.056138004s
FAIL (2025-07-04 12:34:27 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 2.056166017s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
2.081706279s
FAIL (2025-07-04 12:34:03 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 2.081746937s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
1.911020383s
FAIL (2025-07-04 12:33:19 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.911043047s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
1.888402983s
FAIL (2025-07-04 12:32:40 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.888430314s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
1.974033135s
FAIL (2025-07-04 12:31:58 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.974062479s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
1.889168447s
FAIL (2025-07-04 12:30:50 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.889199106s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
1.862776088s
FAIL (2025-07-04 12:29:42 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.862803709s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
1.864461914s
FAIL (2025-07-04 12:29:08 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.864498754s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
1.861812462s
FAIL (2025-07-04 12:28:06 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.86183253s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
1.862659683s
FAIL (2025-07-04 12:26:59 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.862685041s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
1.882154499s
FAIL (2025-07-04 12:26:00 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.882196479s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
1.904681965s
FAIL (2025-07-04 12:24:57 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.90470508s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
1.889045554s
FAIL (2025-07-04 12:24:31 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.889073527s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
1.887934175s
FAIL (2025-07-04 12:24:02 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.887960024s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
1.857637315s
FAIL (2025-07-04 12:22:49 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.85767714s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
1.96822032s
FAIL (2025-07-04 12:22:27 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.96825129s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
1.858602291s
FAIL (2025-07-04 12:21:29 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.858629725s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
1.865155729s
FAIL (2025-07-04 12:20:56 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.865190485s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
1.891824669s
FAIL (2025-07-04 12:20:02 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.891855157s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
1.922960065s
FAIL (2025-07-04 12:18:59 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.922979743s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
1.921680384s
FAIL (2025-07-04 12:18:13 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.921707256s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.46349063s
FAIL (2025-07-04 12:17:23 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 2.463512682s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
1.918779358s
FAIL (2025-07-04 12:16:41 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ing upload Jul 4 12:00:01.328864037 (266h49m36.099505372s) [upload=213357]: Done starting upload Jul 4 12:00:01.328932980 (266h49m36.099574325s) [upload=213357]: Uploading segments... Jul 4 12:00:01.328982184 (266h49m36.099623519s) [upload=213357]: Got next segment. Inline: true Jul 4 12:00:01.328988065 (266h49m36.09962941s) [upload=213357]: Waiting for error group managing segments... Jul 4 12:00:01.329091844 (266h49m36.099733179s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:00:01.329099919 (266h49m36.099741254s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:00:01.329103686 (266h49m36.099745021s) [upload=213357]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:00:01.658020330 (266h49m36.428661665s) [upload=213357]: Done uploading segments... Jul 4 12:00:01.658041821 (266h49m36.428683166s) [upload=213357]: Upload finished. err: <nil> Jul 4 12:01:05.747641042 (266h50m40.518282377s) [upload=213373]: Starting upload Jul 4 12:01:05.747691799 (266h50m40.518333134s) [upload=213373]: Done starting upload Jul 4 12:01:05.747739049 (266h50m40.518380384s) [upload=213373]: Uploading segments... Jul 4 12:01:05.747768185 (266h50m40.51840952s) [upload=213373]: Got next segment. Inline: true Jul 4 12:01:05.747772002 (266h50m40.518413337s) [upload=213373]: Waiting for error group managing segments... Jul 4 12:01:05.747801609 (266h50m40.518442944s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:05.747805908 (266h50m40.518447243s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:05.747808312 (266h50m40.518449647s) [upload=213373]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:06.034238496 (266h50m40.804879831s) [upload=213373]: Done uploading segments... Jul 4 12:01:06.034256941 (266h50m40.804898276s) [upload=213373]: Upload finished. err: <nil> Jul 4 12:01:56.003045287 (266h51m30.773686632s) [upload=213385]: Starting upload Jul 4 12:01:56.003109751 (266h51m30.773751086s) [upload=213385]: Done starting upload Jul 4 12:01:56.003157572 (266h51m30.773798907s) [upload=213385]: Uploading segments... Jul 4 12:01:56.003188962 (266h51m30.773830297s) [upload=213385]: Got next segment. Inline: true Jul 4 12:01:56.003192469 (266h51m30.773833805s) [upload=213385]: Waiting for error group managing segments... Jul 4 12:01:56.003224220 (266h51m30.773865555s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:01:56.003228608 (266h51m30.773869943s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:01:56.003231213 (266h51m30.773872548s) [upload=213385]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:01:56.305189725 (266h51m31.07583106s) [upload=213385]: Done uploading segments... Jul 4 12:01:56.305209392 (266h51m31.075850727s) [upload=213385]: Upload finished. err: <nil> Jul 4 12:02:46.062494652 (266h52m20.833136027s) [upload=213397]: Starting upload Jul 4 12:02:46.062632777 (266h52m20.833274112s) [upload=213397]: Done starting upload Jul 4 12:02:46.062713730 (266h52m20.833355106s) [upload=213397]: Uploading segments... Jul 4 12:02:46.062766411 (266h52m20.833407746s) [upload=213397]: Got next segment. Inline: true Jul 4 12:02:46.062771061 (266h52m20.833412396s) [upload=213397]: Waiting for error group managing segments... Jul 4 12:02:46.062807711 (266h52m20.833449046s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:02:46.062813983 (266h52m20.833455318s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:02:46.062816477 (266h52m20.833457802s) [upload=213397]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:02:46.409590194 (266h52m21.180231529s) [upload=213397]: Done uploading segments... Jul 4 12:02:46.409607427 (266h52m21.180248762s) [upload=213397]: Upload finished. err: <nil> Jul 4 12:03:38.926707844 (266h53m13.697349169s) [upload=213408]: Starting upload Jul 4 12:03:38.926759873 (266h53m13.697401198s) [upload=213408]: Done starting upload Jul 4 12:03:38.926809569 (266h53m13.697450894s) [upload=213408]: Uploading segments... Jul 4 12:03:38.926840207 (266h53m13.697481542s) [upload=213408]: Got next segment. Inline: true Jul 4 12:03:38.926843684 (266h53m13.697485019s) [upload=213408]: Waiting for error group managing segments... Jul 4 12:03:38.926876887 (266h53m13.697518222s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:03:38.926881075 (266h53m13.69752241s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:03:38.926883509 (266h53m13.697524844s) [upload=213408]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:03:39.261711022 (266h53m14.032352357s) [upload=213408]: Done uploading segments... Jul 4 12:03:39.261730990 (266h53m14.032372335s) [upload=213408]: Upload finished. err: <nil> Jul 4 12:04:33.655047227 (266h54m8.425688562s) [upload=213419]: Starting upload Jul 4 12:04:33.655110577 (266h54m8.425751912s) [upload=213419]: Done starting upload Jul 4 12:04:33.655162707 (266h54m8.425804042s) [upload=213419]: Uploading segments... Jul 4 12:04:33.655193406 (266h54m8.425834741s) [upload=213419]: Got next segment. Inline: true Jul 4 12:04:33.655197764 (266h54m8.425839099s) [upload=213419]: Waiting for error group managing segments... Jul 4 12:04:33.655230867 (266h54m8.425872202s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:04:33.655235427 (266h54m8.425876772s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:04:33.655237962 (266h54m8.425879297s) [upload=213419]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:04:33.928276246 (266h54m8.698917581s) [upload=213419]: Done uploading segments... Jul 4 12:04:33.928296615 (266h54m8.69893795s) [upload=213419]: Upload finished. err: <nil> Jul 4 12:05:38.780847631 (266h55m13.551488966s) [upload=213430]: Starting upload Jul 4 12:05:38.780956359 (266h55m13.551597704s) [upload=213430]: Done starting upload Jul 4 12:05:38.781022186 (266h55m13.551663521s) [upload=213430]: Uploading segments... Jul 4 12:05:38.781066089 (266h55m13.551707424s) [upload=213430]: Got next segment. Inline: true Jul 4 12:05:38.781071028 (266h55m13.551712363s) [upload=213430]: Waiting for error group managing segments... Jul 4 12:05:38.781115203 (266h55m13.551756538s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:05:38.781121144 (266h55m13.551762489s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:05:38.781124681 (266h55m13.551766016s) [upload=213430]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:05:39.060927407 (266h55m13.831568742s) [upload=213430]: Done uploading segments... Jul 4 12:05:39.060978975 (266h55m13.83162031s) [upload=213430]: Upload finished. err: <nil> Jul 4 12:06:13.140680947 (266h55m47.911322282s) [upload=213441]: Starting upload Jul 4 12:06:13.140747514 (266h55m47.91138885s) [upload=213441]: Done starting upload Jul 4 12:06:13.140807269 (266h55m47.911448594s) [upload=213441]: Uploading segments... Jul 4 12:06:13.140837216 (266h55m47.911478551s) [upload=213441]: Got next segment. Inline: true Jul 4 12:06:13.140840662 (266h55m47.911481997s) [upload=213441]: Waiting for error group managing segments... Jul 4 12:06:13.140869918 (266h55m47.911511253s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:13.140874196 (266h55m47.911515531s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:13.140876661 (266h55m47.911517996s) [upload=213441]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:13.424724585 (266h55m48.19536592s) [upload=213441]: Done uploading segments... Jul 4 12:06:13.424761004 (266h55m48.195402339s) [upload=213441]: Upload finished. err: <nil> Jul 4 12:06:37.948769780 (266h56m12.719411115s) [upload=213446]: Starting upload Jul 4 12:06:37.948934455 (266h56m12.71957579s) [upload=213446]: Done starting upload Jul 4 12:06:37.949012234 (266h56m12.719653569s) [upload=213446]: Uploading segments... Jul 4 12:06:37.949082248 (266h56m12.719723573s) [upload=213446]: Got next segment. Inline: true Jul 4 12:06:37.949088239 (266h56m12.719729574s) [upload=213446]: Waiting for error group managing segments... Jul 4 12:06:37.949149076 (266h56m12.719790411s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:06:37.949155578 (266h56m12.719796913s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:06:37.949158694 (266h56m12.719800029s) [upload=213446]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:06:38.227278720 (266h56m12.997920055s) [upload=213446]: Done uploading segments... Jul 4 12:06:38.227297677 (266h56m12.997939012s) [upload=213446]: Upload finished. err: <nil> Jul 4 12:07:32.185181939 (266h57m6.955823274s) [upload=213460]: Starting upload Jul 4 12:07:32.185257103 (266h57m6.955898438s) [upload=213460]: Done starting upload Jul 4 12:07:32.185332628 (266h57m6.955973963s) [upload=213460]: Uploading segments... Jul 4 12:07:32.185377733 (266h57m6.956019078s) [upload=213460]: Got next segment. Inline: true Jul 4 12:07:32.185386230 (266h57m6.956027555s) [upload=213460]: Waiting for error group managing segments... Jul 4 12:07:32.185437007 (266h57m6.956078332s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 4 12:07:32.185441937 (266h57m6.956083262s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 4 12:07:32.185444371 (266h57m6.956085706s) [upload=213460]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 4 12:07:32.450383502 (266h57m7.221024827s) [upload=213460]: Done uploading segments... Jul 4 12:07:32.450402248 (266h57m7.221043583s) [upload=213460]: Upload finished. err: <nil>
Duration: 1.918805637s
core
uplink
us-select-1
|