Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.264212037s FAIL (2025-09-07 20:03:44 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.264233578s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.811130974s FAIL (2025-09-07 20:03:39 UTC) orbiter warning: context deadline exceeded Duration: 5.811212151s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.613228694s FAIL (2025-09-07 20:03:15 UTC) orbiter warning: context deadline exceeded Duration: 5.61324822s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.052888274s FAIL (2025-09-07 20:03:05 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.052921358s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.923449811s FAIL (2025-09-07 20:02:26 UTC) orbiter warning: context deadline exceeded Duration: 5.923473546s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.466746915s FAIL (2025-09-07 20:02:23 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.466772043s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.000813156s FAIL (2025-09-07 20:02:17 UTC) orbiter warning: context deadline exceeded Duration: 1m0.000838154s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 40.528078823s FAIL (2025-09-07 20:02:16 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 40.528101267s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.161452632s FAIL (2025-09-07 17:58:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.161476749s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 7.290222819s FAIL (2025-09-07 17:57:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 7.290249841s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 14.146713582s FAIL (2025-09-07 17:57:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 14.146740503s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 7.355655695s FAIL (2025-09-07 17:56:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 7.355681575s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.677940333s FAIL (2025-09-07 17:55:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.677986902s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 12.489631113s FAIL (2025-09-07 17:55:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 12.489660038s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.180260402s FAIL (2025-09-07 17:54:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.180287494s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 9.4439878s FAIL (2025-09-07 17:53:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 9.444011835s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.844109974s FAIL (2025-09-07 17:52:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.844135763s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 7.570177465s FAIL (2025-09-07 17:51:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 7.570202322s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 7.624503366s FAIL (2025-09-07 17:50:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 7.624535007s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 8.48700986s FAIL (2025-09-07 17:50:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 8.487048063s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.064432303s FAIL (2025-09-07 17:49:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.064455107s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 8.931730059s FAIL (2025-09-07 17:49:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 8.931756548s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 11.598922384s FAIL (2025-09-07 17:48:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 11.59895106s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.1801244s FAIL (2025-09-07 17:47:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.18015545s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.323128756s FAIL (2025-09-07 17:47:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.323155989s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.164712985s FAIL (2025-09-07 17:45:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.164735287s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.343508506s FAIL (2025-09-07 17:45:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.34353703s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.002944981s FAIL (2025-09-07 17:43:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.002972163s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 19.606354665s FAIL (2025-09-07 17:43:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 19.606382288s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 9.691634819s FAIL (2025-09-07 17:42:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 9.691657722s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X.....................X.....................................] duration: 13.249866394s FAIL (2025-09-07 17:42:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 13.249897683s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X.....................X....................................] duration: 6.239979751s FAIL (2025-09-07 17:40:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.239997223s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X.X.....................X...................................] duration: 6.0544013s FAIL (2025-09-07 17:40:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.054433793s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X.X.....................X..................................] duration: 6.290130732s FAIL (2025-09-07 17:40:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.290157746s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X.X.....................X.................................] duration: 6.312161314s FAIL (2025-09-07 17:39:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.312201169s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X.X.....................X................................] duration: 6.007855208s FAIL (2025-09-07 17:38:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.007882841s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X.X.....................X...............................] duration: 6.707218269s FAIL (2025-09-07 17:37:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.707248487s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X.X.....................X..............................] duration: 8.049839927s FAIL (2025-09-07 17:35:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 8.049873601s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X.X.....................X.............................] duration: 6.500221969s FAIL (2025-09-07 17:34:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.500258907s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X.X.....................X............................] duration: 6.201084872s FAIL (2025-09-07 17:33:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.201112414s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X.X.....................X...........................] duration: 17.932683783s FAIL (2025-09-07 17:33:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 17.932702709s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X.X.....................X..........................] duration: 6.625309552s FAIL (2025-09-07 17:32:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.625330613s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X.X.....................X.........................] duration: 18.375959219s FAIL (2025-09-07 17:32:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 18.375987523s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X.X.....................X........................] duration: 8.435273407s FAIL (2025-09-07 17:31:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 8.435308625s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X.X.....................X.......................] duration: 22.772331872s FAIL (2025-09-07 17:31:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 22.77235734s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X.X.....................X......................] duration: 6.8502954s FAIL (2025-09-07 17:30:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.850326469s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X.X.....................X.....................] duration: 7.156052218s FAIL (2025-09-07 17:29:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 7.156080441s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X.X.....................X....................] duration: 6.561109613s FAIL (2025-09-07 17:29:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 6.561161031s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X.X.....................X...................] duration: 7.189709073s FAIL (2025-09-07 17:28:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 7.189737727s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X.X.....................X..................] duration: 7.378369057s FAIL (2025-09-07 17:27:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 17:13:45.507189583 (27h45m21.546762703s) [upload=24426]: Starting upload Sep 7 17:13:45.507243646 (27h45m21.546816766s) [upload=24426]: Done starting upload Sep 7 17:13:45.507329561 (27h45m21.546902671s) [upload=24426]: Uploading segments... Sep 7 17:13:45.507444541 (27h45m21.547017661s) [upload=24426]: Got next segment. Inline: false Sep 7 17:13:45.507479497 (27h45m21.547052607s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 17:13:45.507483815 (27h45m21.547056935s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 17:13:45.870819774 (27h45m21.910392884s) [seg_pos={0 0}, upload=24426]: Begin upload segment... Sep 7 17:13:45.870862817 (27h45m21.910435937s) [seg_pos={0 0}, upload=24426]: Done begin upload segment. Sep 7 17:13:45.870885028 (27h45m21.910458148s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.870962938 (27h45m21.910536058s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871265917 (27h45m21.910839037s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871551995 (27h45m21.911125105s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871567594 (27h45m21.911140714s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.871822513 (27h45m21.911395633s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872164036 (27h45m21.911737156s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872467396 (27h45m21.912040516s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872735219 (27h45m21.912308339s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.872975870 (27h45m21.91254899s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873004725 (27h45m21.912577835s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873245676 (27h45m21.912818796s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873320058 (27h45m21.912893178s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873654288 (27h45m21.913227408s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.873951517 (27h45m21.913524637s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:45.874228136 (27h45m21.913801256s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Uploading piece... Sep 7 17:13:47.677610019 (27h45m23.717183139s) [upload=24426]: Next returned nil segment Sep 7 17:13:47.677637462 (27h45m23.717210582s) [upload=24426]: Waiting for error group managing segments... Sep 7 17:13:47.990994583 (27h45m24.030567693s) [noise=true , address=45.91.116.13:4153 , piece_id=VTA4BGLZKHRKX457WWDQVIUACWFRTV6FT2H5ARDXTXYATLF6B2SA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.034159883 (27h45m24.073733003s) [noise=true , address=45.91.116.9:4261 , piece_id=SZ25KIKHHWKDVZLEHREUBYU3OWFKHMZEOLKV54AYASFL7BGSDLNQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.070992824 (27h45m24.110565944s) [noise=true , address=45.91.116.14:4189 , piece_id=OMMEBP6BXXIJK2RHPR5GKU2OLRNXACHPDAHDUVAILIVTE5BY5S7A, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.073390983 (27h45m24.112964103s) [noise=true , address=45.91.116.10:4083 , piece_id=NIDXCAG44BHJVFUD5RXMDQDMHBEC6MUTGWWYDXTYY4RRWADBXAOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.152494830 (27h45m24.19206795s) [noise=true , address=45.91.116.6:4975 , piece_id=FZNREQHOXGQPTCG53KY7ZJXEPE2YYCR57PKWH45ZNCUHZM6NMSOQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.207632549 (27h45m24.247205669s) [noise=true , address=45.91.116.7:4011 , piece_id=BX5NTTZKXKITNV5V7AXP3EQITJO32J7BWMURWV3KBYR7EJYWKROQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.208386362 (27h45m24.247959482s) [noise=true , address=45.91.116.5:4299 , piece_id=KQUOIL4GW3USOLMEDYBXAUAD5VNTYBZSIYXLYHPO6JNWYHH6G6BA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.215157178 (27h45m24.254730298s) [noise=true , address=45.91.116.6:4974 , piece_id=I6652QVNLU6ZE3QWO4YFRSBYXQ423C53EJ6SHZBMUCZSAJMI2RRQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.223393518 (27h45m24.262966638s) [noise=true , address=77.90.31.16:4733 , piece_id=KLOESTXNOWO7PD3WWLBWH6NML6UYZZJYZ56OZTQNAANPCVKDOLHA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.227965647 (27h45m24.267538767s) [noise=true , address=45.91.116.8:4048 , piece_id=ZLJ5F5CWCLCXS6BN3OWZDSWTRDEV5B7VSGK4ZKQ6E5RBY7JSIIWQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.228813529 (27h45m24.268386649s) [noise=true , address=45.91.116.9:4263 , piece_id=FZZ2NDU7SXODHYGM3MITLPOY2OHFNJ4FF2KOB5RIXWW7QZX5DEGQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.231824431 (27h45m24.271397551s) [noise=true , address=45.91.116.13:4154 , piece_id=IOIMHL3TXUCUX2VFHFGXH3NH5A2GXEQK3TXLQY5XBH74PTYNUWDA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404931258 (27h45m24.444504378s) [noise=true , address=45.91.116.12:4225 , piece_id=OVCDI5BISAFK4SEDOISOJP34EDM265VT6FXWUTRECIKIASQRGYBA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: <nil> Sep 7 17:13:48.404958600 (27h45m24.44453171s) [seg_pos={0 0}, upload=24426]: Segment reached optimal threshold of 13 pieces. Sep 7 17:13:48.405106563 (27h45m24.444679723s) [noise=true , address=77.90.31.10:4519 , piece_id=ZCVVOBC2RZUN6JBZPHT2V7OJLJHTVU6WWTLJDJ4GKKXPB22JWOMQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405189482 (27h45m24.444762602s) [noise=true , address=45.91.116.11:4117 , piece_id=LBD4ZQ5VBYA63Q67J2AULCX5KMUIM77533AVTAFV4HPUN7WVXVEQ, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405260468 (27h45m24.444833588s) [noise=true , address=45.91.116.7:4009 , piece_id=V6KUPHHR5M64KXTNQW67YMABQVMET57F7LNJXSPCTLHVJAE4UMEA, seg_pos={0 0}, upload=24426]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 17:13:48.405289704 (27h45m24.444862824s) [seg_pos={0 0}, upload=24426]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 17:13:48.405375648 (27h45m24.444948758s) [upload=24426]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 17:13:48.405379535 (27h45m24.444952655s) [upload=24426]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 17:13:48.714009000 (27h45m24.753582121s) [upload=24426]: Done uploading segments... Sep 7 17:13:48.714037605 (27h45m24.753610725s) [upload=24426]: Upload finished. err: <nil> Duration: 7.378394006s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)