Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.311704175s FAIL (2025-10-07 08:20:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.311746366s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 10.776979221s FAIL (2025-10-07 08:20:04 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 10.77701501s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.361569523s FAIL (2025-10-07 08:19:59 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.361599381s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.508149986s FAIL (2025-10-07 08:19:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.508181506s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.122291624s FAIL (2025-10-07 08:19:18 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.122321312s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.562315929s FAIL (2025-10-07 08:19:16 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 6.562340505s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.277806719s FAIL (2025-10-07 08:19:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.277838669s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.079397614s FAIL (2025-10-07 08:18:56 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.07943217s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.073459629s FAIL (2025-10-07 08:18:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.073488464s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 14.116303866s FAIL (2025-10-07 08:18:27 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 14.116358911s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.906893237s FAIL (2025-10-07 08:18:22 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.906912664s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.346308096s FAIL (2025-10-07 08:18:05 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.346342802s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.212197906s FAIL (2025-10-07 08:17:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.21222633s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.741629658s FAIL (2025-10-07 08:17:20 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.741665116s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.701054868s FAIL (2025-10-07 08:17:10 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.701086579s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.284837799s FAIL (2025-10-07 08:17:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.284912321s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.348310916s FAIL (2025-10-07 08:16:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.348337859s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.240987816s FAIL (2025-10-07 08:16:29 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.241032901s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.026185293s FAIL (2025-10-07 08:15:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 6.026213487s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.047984867s FAIL (2025-10-07 08:15:57 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.048020214s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.153163974s FAIL (2025-10-07 08:15:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.153194762s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.91597547s FAIL (2025-10-07 08:15:16 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.916009364s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.165549797s FAIL (2025-10-07 08:14:51 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.165584934s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.073950916s FAIL (2025-10-07 08:14:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.073980803s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.985450144s FAIL (2025-10-07 08:14:36 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.985504156s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.105317714s FAIL (2025-10-07 08:14:22 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.105366658s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 10.436397904s FAIL (2025-10-07 08:13:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 10.436468149s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 15.490963301s FAIL (2025-10-07 08:13:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 15.490994932s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 25.865926797s FAIL (2025-10-07 08:13:41 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 25.865958959s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.754852945s FAIL (2025-10-07 08:12:48 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.754885107s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.853393936s FAIL (2025-10-07 08:12:47 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.853424042s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.110958416s FAIL (2025-10-07 08:12:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.110983563s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.724025603s FAIL (2025-10-07 08:12:24 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.724070348s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.294517967s FAIL (2025-10-07 08:12:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.294560889s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.794350936s FAIL (2025-10-07 08:11:40 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.794380895s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.843378988s FAIL (2025-10-07 08:11:15 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.843614205s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.129451754s FAIL (2025-10-07 08:11:06 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.129481271s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.10181064s FAIL (2025-10-07 08:11:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.101842941s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.824696878s FAIL (2025-10-07 08:10:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.824730611s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.731288043s FAIL (2025-10-07 08:10:23 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.731324543s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.337038805s FAIL (2025-10-07 08:10:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.337073371s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.844544215s FAIL (2025-10-07 08:09:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.844582366s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.908586602s FAIL (2025-10-07 08:09:41 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.90864862s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.135793751s FAIL (2025-10-07 08:09:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.135825101s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.60436996s FAIL (2025-10-07 08:08:54 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.604464558s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.67944712s FAIL (2025-10-07 08:08:54 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.67947406s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.998589825s FAIL (2025-10-07 08:08:23 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.998623079s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.23889025s FAIL (2025-10-07 08:08:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.238920318s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.644327683s FAIL (2025-10-07 08:08:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.64438936s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.732608507s FAIL (2025-10-07 08:07:58 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.732686436s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)