Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.570990917s FAIL (2025-10-07 08:07:38 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.571035231s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.800479336s FAIL (2025-10-07 08:07:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.800516996s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.107841599s FAIL (2025-10-07 08:07:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.107874131s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.275952155s FAIL (2025-10-07 08:07:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.275983936s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.869072312s FAIL (2025-10-07 08:06:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.86910239s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.85722449s FAIL (2025-10-07 08:06:29 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.857260089s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.327299738s FAIL (2025-10-07 08:06:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.327332108s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.911460666s FAIL (2025-10-07 08:06:03 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.91148987s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.828291465s FAIL (2025-10-07 08:05:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.828373471s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.944292293s FAIL (2025-10-07 08:05:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.944319734s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.20003253s FAIL (2025-10-07 08:05:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.200058399s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.674266884s FAIL (2025-10-07 08:04:56 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.674300278s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.143254204s FAIL (2025-10-07 08:04:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.143282859s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.936253318s FAIL (2025-10-07 08:04:03 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.936292531s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.905723734s FAIL (2025-10-07 08:03:53 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.90575817s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.217085716s FAIL (2025-10-07 08:03:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.21711447s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.032623932s FAIL (2025-10-07 08:03:22 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.03265846s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.951791653s FAIL (2025-10-07 08:03:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.951839925s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.155792929s FAIL (2025-10-07 08:03:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.155821483s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.170672662s FAIL (2025-10-07 08:02:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.170701877s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.165572331s FAIL (2025-10-07 08:02:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.165630582s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.760165051s FAIL (2025-10-07 08:02:15 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.760302753s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.908980493s FAIL (2025-10-07 08:02:04 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.909022643s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.628403667s FAIL (2025-10-07 08:02:00 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.628741461s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.049030845s FAIL (2025-10-07 08:01:23 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 6.049066622s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.088664242s FAIL (2025-10-07 08:01:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.088707715s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.934572905s FAIL (2025-10-07 08:00:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.934610317s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.978146967s FAIL (2025-10-07 08:00:45 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.978276474s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.070442766s FAIL (2025-10-07 08:00:33 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 6.070479065s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.220107284s FAIL (2025-10-07 08:00:17 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.220501385s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.379648672s FAIL (2025-10-07 08:00:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.379682145s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.25293294s FAIL (2025-10-07 08:00:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 6.252972446s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.752653494s FAIL (2025-10-07 07:59:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.752787168s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.280385698s FAIL (2025-10-07 07:59:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.280411508s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.997213276s FAIL (2025-10-07 07:59:40 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.997343413s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.825766968s FAIL (2025-10-07 07:59:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.825812134s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.154063871s FAIL (2025-10-07 07:59:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.154097767s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.813238211s FAIL (2025-10-07 07:59:10 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.8132949s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 7.68850665s FAIL (2025-10-07 07:58:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 7.688538071s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.746054821s FAIL (2025-10-07 07:58:36 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.746087242s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.201490964s FAIL (2025-10-07 07:58:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.201743664s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.283031914s FAIL (2025-10-07 07:57:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.283067382s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.860445031s FAIL (2025-10-07 07:57:50 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.86047658s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.305666495s FAIL (2025-10-07 07:57:47 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.305697945s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.812460079s FAIL (2025-10-07 07:57:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.812490037s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.209346423s FAIL (2025-10-07 07:56:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.209378283s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.141899019s FAIL (2025-10-07 07:56:35 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.142008016s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.86975495s FAIL (2025-10-07 07:56:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.869786228s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.925238089s FAIL (2025-10-07 07:56:09 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.925268346s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.057445246s FAIL (2025-10-07 07:55:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 6.057476355s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)