Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................XXX...........X..........................] duration: 16.644068426s trace: https://jaeger.dev.storj.io/trace/4a055e3bcb21e3db FAIL (2026-01-16 17:33:56 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 16.644102309s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 55 out of 60 is failed (threshold=6) [XXXX.XXXXXXXXXXXXXX.XXXXXXXX.XXXXXXXXXXXXXXXXX.XXXXXX.XXXXXX] duration: 5.000907638s trace: https://jaeger.dev.storj.io/trace/6ec32fa59e0cbb92 FAIL (2026-01-16 17:33:41 UTC) orbiter warning: context deadline exceeded Duration: 5.000927976s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 5.829398597s trace: https://jaeger.dev.storj.io/trace/ff1eaafc0d95989 FAIL (2026-01-16 17:33:38 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 16 16:56:06.586316499 (25h0m3.480025108s) [upload=21929]: Starting upload Jan 16 16:56:06.586371244 (25h0m3.480079853s) [upload=21929]: Done starting upload Jan 16 16:56:06.586529134 (25h0m3.480237763s) [upload=21929]: Uploading segments... Jan 16 16:56:06.586687636 (25h0m3.480396275s) [upload=21929]: Got next segment. Inline: false Jan 16 16:56:06.586750094 (25h0m3.480458723s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 16 16:56:06.586757138 (25h0m3.480465767s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 16 16:56:08.120364931 (25h0m5.01407354s) [seg_pos={0 0}, upload=21929]: Begin upload segment... Jan 16 16:56:08.120413474 (25h0m5.014122083s) [seg_pos={0 0}, upload=21929]: Done begin upload segment. Jan 16 16:56:08.120440615 (25h0m5.014149224s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.120799800 (25h0m5.014508419s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.121695454 (25h0m5.015404084s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.122317739 (25h0m5.016026349s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123161636 (25h0m5.016870245s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123395732 (25h0m5.017104351s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123848494 (25h0m5.017557113s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123868092 (25h0m5.017576731s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124249808 (25h0m5.017958417s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124389284 (25h0m5.018097893s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124619222 (25h0m5.018327831s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124999566 (25h0m5.018708175s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125202182 (25h0m5.018910801s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125576805 (25h0m5.019285414s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125952470 (25h0m5.019661079s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.126492900 (25h0m5.020201509s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:10.730578935 (25h0m7.624287544s) [upload=21929]: Next returned nil segment Jan 16 16:56:10.730594134 (25h0m7.624302753s) [upload=21929]: Waiting for error group managing segments... Jan 16 16:56:11.329683279 (25h0m8.223391888s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.338893335 (25h0m8.232601944s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400223651 (25h0m8.29393226s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400340974 (25h0m8.294049583s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.445450294 (25h0m8.339158904s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.964879212 (25h0m8.858587821s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.027795357 (25h0m8.921503966s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.267543450 (25h0m9.161252059s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.337643945 (25h0m9.231352554s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.380001967 (25h0m9.273710576s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.850164228 (25h0m9.743872837s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.183873887 (25h0m10.077582496s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361354387 (25h0m10.255063026s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361383082 (25h0m10.255091691s) [seg_pos={0 0}, upload=21929]: Satisfied cohort requirements after 13 pieces Jan 16 16:56:13.361615564 (25h0m10.255324173s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361624061 (25h0m10.25533267s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361904004 (25h0m10.255612623s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: read tcp 188.166.225.170:42244->145.79.227.38:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 16 16:56:13.361925344 (25h0m10.255633963s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361971022 (25h0m10.255679631s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361978977 (25h0m10.255687596s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.362041817 (25h0m10.255750426s) [seg_pos={0 0}, upload=21929]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 16 16:56:13.362197272 (25h0m10.255905881s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 16 16:56:13.362205868 (25h0m10.255914477s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 16 16:56:13.965177888 (25h0m10.858886497s) [upload=21929]: Done uploading segments... Jan 16 16:56:13.965204158 (25h0m10.858912767s) [upload=21929]: Upload finished. err: <nil> Duration: 5.829443172s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................XXX...........X.........................] duration: 14.393416142s trace: https://jaeger.dev.storj.io/trace/4a055e3bcb21e3db FAIL (2026-01-16 17:33:27 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 14.393442354s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 55 out of 60 is failed (threshold=6) [XXXXX.XXXXXXXXXXXXXX.XXXXXXXX.XXXXXXXXXXXXXXXXX.XXXXXX.XXXXX] duration: 5.000914368s trace: https://jaeger.dev.storj.io/trace/5011ab0a242f60d5 FAIL (2026-01-16 17:33:21 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.0009363s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 13.391049907s trace: https://jaeger.dev.storj.io/trace/70c2b0731de5fb9c FAIL (2026-01-16 17:33:08 UTC) orbiter warning: context deadline exceeded Duration: 13.391078551s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 5.737670441s trace: https://jaeger.dev.storj.io/trace/ff1eaafc0d95989 FAIL (2026-01-16 17:32:48 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 16 16:56:06.586316499 (25h0m3.480025108s) [upload=21929]: Starting upload Jan 16 16:56:06.586371244 (25h0m3.480079853s) [upload=21929]: Done starting upload Jan 16 16:56:06.586529134 (25h0m3.480237763s) [upload=21929]: Uploading segments... Jan 16 16:56:06.586687636 (25h0m3.480396275s) [upload=21929]: Got next segment. Inline: false Jan 16 16:56:06.586750094 (25h0m3.480458723s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 16 16:56:06.586757138 (25h0m3.480465767s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 16 16:56:08.120364931 (25h0m5.01407354s) [seg_pos={0 0}, upload=21929]: Begin upload segment... Jan 16 16:56:08.120413474 (25h0m5.014122083s) [seg_pos={0 0}, upload=21929]: Done begin upload segment. Jan 16 16:56:08.120440615 (25h0m5.014149224s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.120799800 (25h0m5.014508419s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.121695454 (25h0m5.015404084s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.122317739 (25h0m5.016026349s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123161636 (25h0m5.016870245s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123395732 (25h0m5.017104351s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123848494 (25h0m5.017557113s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123868092 (25h0m5.017576731s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124249808 (25h0m5.017958417s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124389284 (25h0m5.018097893s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124619222 (25h0m5.018327831s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124999566 (25h0m5.018708175s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125202182 (25h0m5.018910801s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125576805 (25h0m5.019285414s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125952470 (25h0m5.019661079s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.126492900 (25h0m5.020201509s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:10.730578935 (25h0m7.624287544s) [upload=21929]: Next returned nil segment Jan 16 16:56:10.730594134 (25h0m7.624302753s) [upload=21929]: Waiting for error group managing segments... Jan 16 16:56:11.329683279 (25h0m8.223391888s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.338893335 (25h0m8.232601944s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400223651 (25h0m8.29393226s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400340974 (25h0m8.294049583s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.445450294 (25h0m8.339158904s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.964879212 (25h0m8.858587821s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.027795357 (25h0m8.921503966s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.267543450 (25h0m9.161252059s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.337643945 (25h0m9.231352554s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.380001967 (25h0m9.273710576s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.850164228 (25h0m9.743872837s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.183873887 (25h0m10.077582496s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361354387 (25h0m10.255063026s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361383082 (25h0m10.255091691s) [seg_pos={0 0}, upload=21929]: Satisfied cohort requirements after 13 pieces Jan 16 16:56:13.361615564 (25h0m10.255324173s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361624061 (25h0m10.25533267s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361904004 (25h0m10.255612623s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: read tcp 188.166.225.170:42244->145.79.227.38:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 16 16:56:13.361925344 (25h0m10.255633963s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361971022 (25h0m10.255679631s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361978977 (25h0m10.255687596s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.362041817 (25h0m10.255750426s) [seg_pos={0 0}, upload=21929]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 16 16:56:13.362197272 (25h0m10.255905881s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 16 16:56:13.362205868 (25h0m10.255914477s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 16 16:56:13.965177888 (25h0m10.858886497s) [upload=21929]: Done uploading segments... Jan 16 16:56:13.965204158 (25h0m10.858912767s) [upload=21929]: Upload finished. err: <nil> Duration: 5.737699077s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................XXX...........X........................] duration: 17.278134478s trace: https://jaeger.dev.storj.io/trace/4a055e3bcb21e3db FAIL (2026-01-16 17:32:27 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 17.278165237s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 55 out of 60 is failed (threshold=6) [XXXXXX.XXXXXXXXXXXXXX.XXXXXXXX.XXXXXXXXXXXXXXXXX.XXXXXX.XXXX] duration: 5.000411625s trace: https://jaeger.dev.storj.io/trace/5eebc473cb868688 FAIL (2026-01-16 17:32:17 UTC) orbiter warning: context deadline exceeded Duration: 5.000426433s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 25.504458625s trace: https://jaeger.dev.storj.io/trace/70c2b0731de5fb9c FAIL (2026-01-16 17:32:15 UTC) orbiter warning: context deadline exceeded Duration: 25.504483393s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 5.757670484s trace: https://jaeger.dev.storj.io/trace/ff1eaafc0d95989 FAIL (2026-01-16 17:32:11 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 16 16:56:06.586316499 (25h0m3.480025108s) [upload=21929]: Starting upload Jan 16 16:56:06.586371244 (25h0m3.480079853s) [upload=21929]: Done starting upload Jan 16 16:56:06.586529134 (25h0m3.480237763s) [upload=21929]: Uploading segments... Jan 16 16:56:06.586687636 (25h0m3.480396275s) [upload=21929]: Got next segment. Inline: false Jan 16 16:56:06.586750094 (25h0m3.480458723s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 16 16:56:06.586757138 (25h0m3.480465767s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 16 16:56:08.120364931 (25h0m5.01407354s) [seg_pos={0 0}, upload=21929]: Begin upload segment... Jan 16 16:56:08.120413474 (25h0m5.014122083s) [seg_pos={0 0}, upload=21929]: Done begin upload segment. Jan 16 16:56:08.120440615 (25h0m5.014149224s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.120799800 (25h0m5.014508419s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.121695454 (25h0m5.015404084s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.122317739 (25h0m5.016026349s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123161636 (25h0m5.016870245s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123395732 (25h0m5.017104351s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123848494 (25h0m5.017557113s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123868092 (25h0m5.017576731s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124249808 (25h0m5.017958417s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124389284 (25h0m5.018097893s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124619222 (25h0m5.018327831s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124999566 (25h0m5.018708175s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125202182 (25h0m5.018910801s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125576805 (25h0m5.019285414s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125952470 (25h0m5.019661079s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.126492900 (25h0m5.020201509s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:10.730578935 (25h0m7.624287544s) [upload=21929]: Next returned nil segment Jan 16 16:56:10.730594134 (25h0m7.624302753s) [upload=21929]: Waiting for error group managing segments... Jan 16 16:56:11.329683279 (25h0m8.223391888s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.338893335 (25h0m8.232601944s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400223651 (25h0m8.29393226s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400340974 (25h0m8.294049583s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.445450294 (25h0m8.339158904s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.964879212 (25h0m8.858587821s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.027795357 (25h0m8.921503966s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.267543450 (25h0m9.161252059s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.337643945 (25h0m9.231352554s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.380001967 (25h0m9.273710576s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.850164228 (25h0m9.743872837s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.183873887 (25h0m10.077582496s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361354387 (25h0m10.255063026s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361383082 (25h0m10.255091691s) [seg_pos={0 0}, upload=21929]: Satisfied cohort requirements after 13 pieces Jan 16 16:56:13.361615564 (25h0m10.255324173s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361624061 (25h0m10.25533267s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361904004 (25h0m10.255612623s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: read tcp 188.166.225.170:42244->145.79.227.38:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 16 16:56:13.361925344 (25h0m10.255633963s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361971022 (25h0m10.255679631s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361978977 (25h0m10.255687596s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.362041817 (25h0m10.255750426s) [seg_pos={0 0}, upload=21929]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 16 16:56:13.362197272 (25h0m10.255905881s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 16 16:56:13.362205868 (25h0m10.255914477s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 16 16:56:13.965177888 (25h0m10.858886497s) [upload=21929]: Done uploading segments... Jan 16 16:56:13.965204158 (25h0m10.858912767s) [upload=21929]: Upload finished. err: <nil> Duration: 5.757699189s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................XXX...........X.......................] duration: 23.098463s trace: https://jaeger.dev.storj.io/trace/4a055e3bcb21e3db FAIL (2026-01-16 17:32:06 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 23.098492696s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 55 out of 60 is failed (threshold=6) [XXXXXXX.XXXXXXXXXXXXXX.XXXXXXXX.XXXXXXXXXXXXXXXXX.XXXXXX.XXX] duration: 5.000701963s trace: https://jaeger.dev.storj.io/trace/da5ccd1a61eb300 FAIL (2026-01-16 17:31:52 UTC) orbiter warning: context deadline exceeded Duration: 5.000725278s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 5.649269766s trace: https://jaeger.dev.storj.io/trace/ff1eaafc0d95989 FAIL (2026-01-16 17:31:38 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 16 16:56:06.586316499 (25h0m3.480025108s) [upload=21929]: Starting upload Jan 16 16:56:06.586371244 (25h0m3.480079853s) [upload=21929]: Done starting upload Jan 16 16:56:06.586529134 (25h0m3.480237763s) [upload=21929]: Uploading segments... Jan 16 16:56:06.586687636 (25h0m3.480396275s) [upload=21929]: Got next segment. Inline: false Jan 16 16:56:06.586750094 (25h0m3.480458723s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 16 16:56:06.586757138 (25h0m3.480465767s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 16 16:56:08.120364931 (25h0m5.01407354s) [seg_pos={0 0}, upload=21929]: Begin upload segment... Jan 16 16:56:08.120413474 (25h0m5.014122083s) [seg_pos={0 0}, upload=21929]: Done begin upload segment. Jan 16 16:56:08.120440615 (25h0m5.014149224s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.120799800 (25h0m5.014508419s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.121695454 (25h0m5.015404084s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.122317739 (25h0m5.016026349s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123161636 (25h0m5.016870245s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123395732 (25h0m5.017104351s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123848494 (25h0m5.017557113s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123868092 (25h0m5.017576731s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124249808 (25h0m5.017958417s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124389284 (25h0m5.018097893s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124619222 (25h0m5.018327831s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124999566 (25h0m5.018708175s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125202182 (25h0m5.018910801s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125576805 (25h0m5.019285414s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125952470 (25h0m5.019661079s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.126492900 (25h0m5.020201509s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:10.730578935 (25h0m7.624287544s) [upload=21929]: Next returned nil segment Jan 16 16:56:10.730594134 (25h0m7.624302753s) [upload=21929]: Waiting for error group managing segments... Jan 16 16:56:11.329683279 (25h0m8.223391888s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.338893335 (25h0m8.232601944s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400223651 (25h0m8.29393226s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400340974 (25h0m8.294049583s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.445450294 (25h0m8.339158904s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.964879212 (25h0m8.858587821s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.027795357 (25h0m8.921503966s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.267543450 (25h0m9.161252059s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.337643945 (25h0m9.231352554s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.380001967 (25h0m9.273710576s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.850164228 (25h0m9.743872837s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.183873887 (25h0m10.077582496s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361354387 (25h0m10.255063026s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361383082 (25h0m10.255091691s) [seg_pos={0 0}, upload=21929]: Satisfied cohort requirements after 13 pieces Jan 16 16:56:13.361615564 (25h0m10.255324173s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361624061 (25h0m10.25533267s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361904004 (25h0m10.255612623s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: read tcp 188.166.225.170:42244->145.79.227.38:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 16 16:56:13.361925344 (25h0m10.255633963s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361971022 (25h0m10.255679631s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361978977 (25h0m10.255687596s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.362041817 (25h0m10.255750426s) [seg_pos={0 0}, upload=21929]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 16 16:56:13.362197272 (25h0m10.255905881s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 16 16:56:13.362205868 (25h0m10.255914477s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 16 16:56:13.965177888 (25h0m10.858886497s) [upload=21929]: Done uploading segments... Jan 16 16:56:13.965204158 (25h0m10.858912767s) [upload=21929]: Upload finished. err: <nil> Duration: 5.649299212s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 54 out of 60 is failed (threshold=6) [.XXXXXXX.XXXXXXXXXXXXXX.XXXXXXXX.XXXXXXXXXXXXXXXXX.XXXXXX.XX] duration: 5.001604573s trace: https://jaeger.dev.storj.io/trace/2308bb1c5cb67e5 FAIL (2026-01-16 17:31:31 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001625584s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 5.821270925s trace: https://jaeger.dev.storj.io/trace/ff1eaafc0d95989 FAIL (2026-01-16 17:31:08 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 16 16:56:06.586316499 (25h0m3.480025108s) [upload=21929]: Starting upload Jan 16 16:56:06.586371244 (25h0m3.480079853s) [upload=21929]: Done starting upload Jan 16 16:56:06.586529134 (25h0m3.480237763s) [upload=21929]: Uploading segments... Jan 16 16:56:06.586687636 (25h0m3.480396275s) [upload=21929]: Got next segment. Inline: false Jan 16 16:56:06.586750094 (25h0m3.480458723s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 16 16:56:06.586757138 (25h0m3.480465767s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 16 16:56:08.120364931 (25h0m5.01407354s) [seg_pos={0 0}, upload=21929]: Begin upload segment... Jan 16 16:56:08.120413474 (25h0m5.014122083s) [seg_pos={0 0}, upload=21929]: Done begin upload segment. Jan 16 16:56:08.120440615 (25h0m5.014149224s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.120799800 (25h0m5.014508419s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.121695454 (25h0m5.015404084s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.122317739 (25h0m5.016026349s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123161636 (25h0m5.016870245s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123395732 (25h0m5.017104351s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123848494 (25h0m5.017557113s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123868092 (25h0m5.017576731s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124249808 (25h0m5.017958417s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124389284 (25h0m5.018097893s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124619222 (25h0m5.018327831s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124999566 (25h0m5.018708175s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125202182 (25h0m5.018910801s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125576805 (25h0m5.019285414s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125952470 (25h0m5.019661079s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.126492900 (25h0m5.020201509s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:10.730578935 (25h0m7.624287544s) [upload=21929]: Next returned nil segment Jan 16 16:56:10.730594134 (25h0m7.624302753s) [upload=21929]: Waiting for error group managing segments... Jan 16 16:56:11.329683279 (25h0m8.223391888s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.338893335 (25h0m8.232601944s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400223651 (25h0m8.29393226s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400340974 (25h0m8.294049583s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.445450294 (25h0m8.339158904s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.964879212 (25h0m8.858587821s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.027795357 (25h0m8.921503966s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.267543450 (25h0m9.161252059s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.337643945 (25h0m9.231352554s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.380001967 (25h0m9.273710576s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.850164228 (25h0m9.743872837s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.183873887 (25h0m10.077582496s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361354387 (25h0m10.255063026s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361383082 (25h0m10.255091691s) [seg_pos={0 0}, upload=21929]: Satisfied cohort requirements after 13 pieces Jan 16 16:56:13.361615564 (25h0m10.255324173s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361624061 (25h0m10.25533267s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361904004 (25h0m10.255612623s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: read tcp 188.166.225.170:42244->145.79.227.38:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 16 16:56:13.361925344 (25h0m10.255633963s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361971022 (25h0m10.255679631s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361978977 (25h0m10.255687596s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.362041817 (25h0m10.255750426s) [seg_pos={0 0}, upload=21929]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 16 16:56:13.362197272 (25h0m10.255905881s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 16 16:56:13.362205868 (25h0m10.255914477s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 16 16:56:13.965177888 (25h0m10.858886497s) [upload=21929]: Done uploading segments... Jan 16 16:56:13.965204158 (25h0m10.858912767s) [upload=21929]: Upload finished. err: <nil> Duration: 5.821298637s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................XXX...........X......................] duration: 16.484989319s trace: https://jaeger.dev.storj.io/trace/4a055e3bcb21e3db FAIL (2026-01-16 17:31:05 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 16.485016401s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 20.009770451s trace: https://jaeger.dev.storj.io/trace/70c2b0731de5fb9c FAIL (2026-01-16 17:31:02 UTC) orbiter warning: context deadline exceeded Duration: 20.009820336s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 53 out of 60 is failed (threshold=6) [..XXXXXXX.XXXXXXXXXXXXXX.XXXXXXXX.XXXXXXXXXXXXXXXXX.XXXXXX.X] duration: 5.001631826s trace: https://jaeger.dev.storj.io/trace/772f8417360d595 FAIL (2026-01-16 17:30:50 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001652516s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 5.757919882s trace: https://jaeger.dev.storj.io/trace/ff1eaafc0d95989 FAIL (2026-01-16 17:30:44 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 16 16:56:06.586316499 (25h0m3.480025108s) [upload=21929]: Starting upload Jan 16 16:56:06.586371244 (25h0m3.480079853s) [upload=21929]: Done starting upload Jan 16 16:56:06.586529134 (25h0m3.480237763s) [upload=21929]: Uploading segments... Jan 16 16:56:06.586687636 (25h0m3.480396275s) [upload=21929]: Got next segment. Inline: false Jan 16 16:56:06.586750094 (25h0m3.480458723s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 16 16:56:06.586757138 (25h0m3.480465767s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 16 16:56:08.120364931 (25h0m5.01407354s) [seg_pos={0 0}, upload=21929]: Begin upload segment... Jan 16 16:56:08.120413474 (25h0m5.014122083s) [seg_pos={0 0}, upload=21929]: Done begin upload segment. Jan 16 16:56:08.120440615 (25h0m5.014149224s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.120799800 (25h0m5.014508419s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.121695454 (25h0m5.015404084s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.122317739 (25h0m5.016026349s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123161636 (25h0m5.016870245s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123395732 (25h0m5.017104351s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123848494 (25h0m5.017557113s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123868092 (25h0m5.017576731s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124249808 (25h0m5.017958417s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124389284 (25h0m5.018097893s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124619222 (25h0m5.018327831s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124999566 (25h0m5.018708175s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125202182 (25h0m5.018910801s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125576805 (25h0m5.019285414s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125952470 (25h0m5.019661079s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.126492900 (25h0m5.020201509s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:10.730578935 (25h0m7.624287544s) [upload=21929]: Next returned nil segment Jan 16 16:56:10.730594134 (25h0m7.624302753s) [upload=21929]: Waiting for error group managing segments... Jan 16 16:56:11.329683279 (25h0m8.223391888s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.338893335 (25h0m8.232601944s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400223651 (25h0m8.29393226s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400340974 (25h0m8.294049583s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.445450294 (25h0m8.339158904s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.964879212 (25h0m8.858587821s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.027795357 (25h0m8.921503966s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.267543450 (25h0m9.161252059s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.337643945 (25h0m9.231352554s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.380001967 (25h0m9.273710576s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.850164228 (25h0m9.743872837s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.183873887 (25h0m10.077582496s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361354387 (25h0m10.255063026s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361383082 (25h0m10.255091691s) [seg_pos={0 0}, upload=21929]: Satisfied cohort requirements after 13 pieces Jan 16 16:56:13.361615564 (25h0m10.255324173s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361624061 (25h0m10.25533267s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361904004 (25h0m10.255612623s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: read tcp 188.166.225.170:42244->145.79.227.38:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 16 16:56:13.361925344 (25h0m10.255633963s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361971022 (25h0m10.255679631s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361978977 (25h0m10.255687596s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.362041817 (25h0m10.255750426s) [seg_pos={0 0}, upload=21929]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 16 16:56:13.362197272 (25h0m10.255905881s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 16 16:56:13.362205868 (25h0m10.255914477s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 16 16:56:13.965177888 (25h0m10.858886497s) [upload=21929]: Done uploading segments... Jan 16 16:56:13.965204158 (25h0m10.858912767s) [upload=21929]: Upload finished. err: <nil> Duration: 5.757944669s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 15.329016462s trace: https://jaeger.dev.storj.io/trace/70c2b0731de5fb9c FAIL (2026-01-16 17:30:40 UTC) orbiter warning: context deadline exceeded Duration: 15.329043995s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................XXX...........X.....................] duration: 23.774317633s trace: https://jaeger.dev.storj.io/trace/4a055e3bcb21e3db FAIL (2026-01-16 17:30:08 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 23.774346248s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 5.804596255s trace: https://jaeger.dev.storj.io/trace/ff1eaafc0d95989 FAIL (2026-01-16 17:30:03 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 16 16:56:06.586316499 (25h0m3.480025108s) [upload=21929]: Starting upload Jan 16 16:56:06.586371244 (25h0m3.480079853s) [upload=21929]: Done starting upload Jan 16 16:56:06.586529134 (25h0m3.480237763s) [upload=21929]: Uploading segments... Jan 16 16:56:06.586687636 (25h0m3.480396275s) [upload=21929]: Got next segment. Inline: false Jan 16 16:56:06.586750094 (25h0m3.480458723s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 16 16:56:06.586757138 (25h0m3.480465767s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 16 16:56:08.120364931 (25h0m5.01407354s) [seg_pos={0 0}, upload=21929]: Begin upload segment... Jan 16 16:56:08.120413474 (25h0m5.014122083s) [seg_pos={0 0}, upload=21929]: Done begin upload segment. Jan 16 16:56:08.120440615 (25h0m5.014149224s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.120799800 (25h0m5.014508419s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.121695454 (25h0m5.015404084s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.122317739 (25h0m5.016026349s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123161636 (25h0m5.016870245s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123395732 (25h0m5.017104351s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123848494 (25h0m5.017557113s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123868092 (25h0m5.017576731s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124249808 (25h0m5.017958417s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124389284 (25h0m5.018097893s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124619222 (25h0m5.018327831s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124999566 (25h0m5.018708175s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125202182 (25h0m5.018910801s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125576805 (25h0m5.019285414s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125952470 (25h0m5.019661079s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.126492900 (25h0m5.020201509s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:10.730578935 (25h0m7.624287544s) [upload=21929]: Next returned nil segment Jan 16 16:56:10.730594134 (25h0m7.624302753s) [upload=21929]: Waiting for error group managing segments... Jan 16 16:56:11.329683279 (25h0m8.223391888s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.338893335 (25h0m8.232601944s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400223651 (25h0m8.29393226s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400340974 (25h0m8.294049583s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.445450294 (25h0m8.339158904s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.964879212 (25h0m8.858587821s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.027795357 (25h0m8.921503966s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.267543450 (25h0m9.161252059s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.337643945 (25h0m9.231352554s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.380001967 (25h0m9.273710576s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.850164228 (25h0m9.743872837s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.183873887 (25h0m10.077582496s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361354387 (25h0m10.255063026s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361383082 (25h0m10.255091691s) [seg_pos={0 0}, upload=21929]: Satisfied cohort requirements after 13 pieces Jan 16 16:56:13.361615564 (25h0m10.255324173s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361624061 (25h0m10.25533267s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361904004 (25h0m10.255612623s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: read tcp 188.166.225.170:42244->145.79.227.38:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 16 16:56:13.361925344 (25h0m10.255633963s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361971022 (25h0m10.255679631s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361978977 (25h0m10.255687596s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.362041817 (25h0m10.255750426s) [seg_pos={0 0}, upload=21929]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 16 16:56:13.362197272 (25h0m10.255905881s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 16 16:56:13.362205868 (25h0m10.255914477s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 16 16:56:13.965177888 (25h0m10.858886497s) [upload=21929]: Done uploading segments... Jan 16 16:56:13.965204158 (25h0m10.858912767s) [upload=21929]: Upload finished. err: <nil> Duration: 5.804626443s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 53 out of 60 is failed (threshold=6) [X..XXXXXXX.XXXXXXXXXXXXXX.XXXXXXXX.XXXXXXXXXXXXXXXXX.XXXXXX.] duration: 3.768531525s trace: https://jaeger.dev.storj.io/trace/b6cd0528e98a2b5 FAIL (2026-01-16 17:29:42 UTC) orbiter warning: context deadline exceeded Duration: 3.768546685s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 17.115689492s trace: https://jaeger.dev.storj.io/trace/70c2b0731de5fb9c FAIL (2026-01-16 17:29:34 UTC) orbiter warning: context deadline exceeded Duration: 17.115755629s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 5.784397797s trace: https://jaeger.dev.storj.io/trace/ff1eaafc0d95989 FAIL (2026-01-16 17:29:30 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 16 16:56:06.586316499 (25h0m3.480025108s) [upload=21929]: Starting upload Jan 16 16:56:06.586371244 (25h0m3.480079853s) [upload=21929]: Done starting upload Jan 16 16:56:06.586529134 (25h0m3.480237763s) [upload=21929]: Uploading segments... Jan 16 16:56:06.586687636 (25h0m3.480396275s) [upload=21929]: Got next segment. Inline: false Jan 16 16:56:06.586750094 (25h0m3.480458723s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 16 16:56:06.586757138 (25h0m3.480465767s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 16 16:56:08.120364931 (25h0m5.01407354s) [seg_pos={0 0}, upload=21929]: Begin upload segment... Jan 16 16:56:08.120413474 (25h0m5.014122083s) [seg_pos={0 0}, upload=21929]: Done begin upload segment. Jan 16 16:56:08.120440615 (25h0m5.014149224s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.120799800 (25h0m5.014508419s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.121695454 (25h0m5.015404084s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.122317739 (25h0m5.016026349s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123161636 (25h0m5.016870245s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123395732 (25h0m5.017104351s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123848494 (25h0m5.017557113s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123868092 (25h0m5.017576731s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124249808 (25h0m5.017958417s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124389284 (25h0m5.018097893s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124619222 (25h0m5.018327831s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124999566 (25h0m5.018708175s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125202182 (25h0m5.018910801s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125576805 (25h0m5.019285414s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125952470 (25h0m5.019661079s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.126492900 (25h0m5.020201509s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:10.730578935 (25h0m7.624287544s) [upload=21929]: Next returned nil segment Jan 16 16:56:10.730594134 (25h0m7.624302753s) [upload=21929]: Waiting for error group managing segments... Jan 16 16:56:11.329683279 (25h0m8.223391888s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.338893335 (25h0m8.232601944s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400223651 (25h0m8.29393226s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400340974 (25h0m8.294049583s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.445450294 (25h0m8.339158904s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.964879212 (25h0m8.858587821s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.027795357 (25h0m8.921503966s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.267543450 (25h0m9.161252059s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.337643945 (25h0m9.231352554s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.380001967 (25h0m9.273710576s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.850164228 (25h0m9.743872837s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.183873887 (25h0m10.077582496s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361354387 (25h0m10.255063026s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361383082 (25h0m10.255091691s) [seg_pos={0 0}, upload=21929]: Satisfied cohort requirements after 13 pieces Jan 16 16:56:13.361615564 (25h0m10.255324173s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361624061 (25h0m10.25533267s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361904004 (25h0m10.255612623s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: read tcp 188.166.225.170:42244->145.79.227.38:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 16 16:56:13.361925344 (25h0m10.255633963s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361971022 (25h0m10.255679631s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361978977 (25h0m10.255687596s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.362041817 (25h0m10.255750426s) [seg_pos={0 0}, upload=21929]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 16 16:56:13.362197272 (25h0m10.255905881s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 16 16:56:13.362205868 (25h0m10.255914477s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 16 16:56:13.965177888 (25h0m10.858886497s) [upload=21929]: Done uploading segments... Jan 16 16:56:13.965204158 (25h0m10.858912767s) [upload=21929]: Upload finished. err: <nil> Duration: 5.784429266s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 53 out of 60 is failed (threshold=6) [.X..XXXXXXX.XXXXXXXXXXXXXX.XXXXXXXX.XXXXXXXXXXXXXXXXX.XXXXXX] duration: 5.000532652s trace: https://jaeger.dev.storj.io/trace/b6cd0528e98a2b5 FAIL (2026-01-16 17:29:13 UTC) orbiter warning: context deadline exceeded Duration: 5.000551619s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 5.851089493s trace: https://jaeger.dev.storj.io/trace/ff1eaafc0d95989 FAIL (2026-01-16 17:29:04 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 16 16:56:06.586316499 (25h0m3.480025108s) [upload=21929]: Starting upload Jan 16 16:56:06.586371244 (25h0m3.480079853s) [upload=21929]: Done starting upload Jan 16 16:56:06.586529134 (25h0m3.480237763s) [upload=21929]: Uploading segments... Jan 16 16:56:06.586687636 (25h0m3.480396275s) [upload=21929]: Got next segment. Inline: false Jan 16 16:56:06.586750094 (25h0m3.480458723s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 16 16:56:06.586757138 (25h0m3.480465767s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 16 16:56:08.120364931 (25h0m5.01407354s) [seg_pos={0 0}, upload=21929]: Begin upload segment... Jan 16 16:56:08.120413474 (25h0m5.014122083s) [seg_pos={0 0}, upload=21929]: Done begin upload segment. Jan 16 16:56:08.120440615 (25h0m5.014149224s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.120799800 (25h0m5.014508419s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.121695454 (25h0m5.015404084s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.122317739 (25h0m5.016026349s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123161636 (25h0m5.016870245s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123395732 (25h0m5.017104351s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123848494 (25h0m5.017557113s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123868092 (25h0m5.017576731s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124249808 (25h0m5.017958417s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124389284 (25h0m5.018097893s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124619222 (25h0m5.018327831s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124999566 (25h0m5.018708175s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125202182 (25h0m5.018910801s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125576805 (25h0m5.019285414s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125952470 (25h0m5.019661079s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.126492900 (25h0m5.020201509s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:10.730578935 (25h0m7.624287544s) [upload=21929]: Next returned nil segment Jan 16 16:56:10.730594134 (25h0m7.624302753s) [upload=21929]: Waiting for error group managing segments... Jan 16 16:56:11.329683279 (25h0m8.223391888s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.338893335 (25h0m8.232601944s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400223651 (25h0m8.29393226s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400340974 (25h0m8.294049583s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.445450294 (25h0m8.339158904s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.964879212 (25h0m8.858587821s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.027795357 (25h0m8.921503966s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.267543450 (25h0m9.161252059s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.337643945 (25h0m9.231352554s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.380001967 (25h0m9.273710576s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.850164228 (25h0m9.743872837s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.183873887 (25h0m10.077582496s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361354387 (25h0m10.255063026s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361383082 (25h0m10.255091691s) [seg_pos={0 0}, upload=21929]: Satisfied cohort requirements after 13 pieces Jan 16 16:56:13.361615564 (25h0m10.255324173s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361624061 (25h0m10.25533267s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361904004 (25h0m10.255612623s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: read tcp 188.166.225.170:42244->145.79.227.38:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 16 16:56:13.361925344 (25h0m10.255633963s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361971022 (25h0m10.255679631s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361978977 (25h0m10.255687596s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.362041817 (25h0m10.255750426s) [seg_pos={0 0}, upload=21929]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 16 16:56:13.362197272 (25h0m10.255905881s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 16 16:56:13.362205868 (25h0m10.255914477s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 16 16:56:13.965177888 (25h0m10.858886497s) [upload=21929]: Done uploading segments... Jan 16 16:56:13.965204158 (25h0m10.858912767s) [upload=21929]: Upload finished. err: <nil> Duration: 5.85111464s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................XXX...........X....................] duration: 20.051009617s trace: https://jaeger.dev.storj.io/trace/4a055e3bcb21e3db FAIL (2026-01-16 17:28:59 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 20.051043271s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 53 out of 60 is failed (threshold=6) [X.X..XXXXXXX.XXXXXXXXXXXXXX.XXXXXXXX.XXXXXXXXXXXXXXXXX.XXXXX] duration: 5.001547215s trace: https://jaeger.dev.storj.io/trace/3f45eeef8ee80511 FAIL (2026-01-16 17:28:41 UTC) orbiter warning: context deadline exceeded Duration: 5.001571873s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................XXX...........X...................] duration: 18.954495697s trace: https://jaeger.dev.storj.io/trace/4a055e3bcb21e3db FAIL (2026-01-16 17:28:32 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 18.954609575s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 5.758768006s trace: https://jaeger.dev.storj.io/trace/ff1eaafc0d95989 FAIL (2026-01-16 17:28:26 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 16 16:56:06.586316499 (25h0m3.480025108s) [upload=21929]: Starting upload Jan 16 16:56:06.586371244 (25h0m3.480079853s) [upload=21929]: Done starting upload Jan 16 16:56:06.586529134 (25h0m3.480237763s) [upload=21929]: Uploading segments... Jan 16 16:56:06.586687636 (25h0m3.480396275s) [upload=21929]: Got next segment. Inline: false Jan 16 16:56:06.586750094 (25h0m3.480458723s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 16 16:56:06.586757138 (25h0m3.480465767s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 16 16:56:08.120364931 (25h0m5.01407354s) [seg_pos={0 0}, upload=21929]: Begin upload segment... Jan 16 16:56:08.120413474 (25h0m5.014122083s) [seg_pos={0 0}, upload=21929]: Done begin upload segment. Jan 16 16:56:08.120440615 (25h0m5.014149224s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.120799800 (25h0m5.014508419s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.121695454 (25h0m5.015404084s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.122317739 (25h0m5.016026349s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123161636 (25h0m5.016870245s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123395732 (25h0m5.017104351s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123848494 (25h0m5.017557113s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123868092 (25h0m5.017576731s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124249808 (25h0m5.017958417s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124389284 (25h0m5.018097893s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124619222 (25h0m5.018327831s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124999566 (25h0m5.018708175s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125202182 (25h0m5.018910801s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125576805 (25h0m5.019285414s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125952470 (25h0m5.019661079s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.126492900 (25h0m5.020201509s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:10.730578935 (25h0m7.624287544s) [upload=21929]: Next returned nil segment Jan 16 16:56:10.730594134 (25h0m7.624302753s) [upload=21929]: Waiting for error group managing segments... Jan 16 16:56:11.329683279 (25h0m8.223391888s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.338893335 (25h0m8.232601944s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400223651 (25h0m8.29393226s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400340974 (25h0m8.294049583s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.445450294 (25h0m8.339158904s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.964879212 (25h0m8.858587821s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.027795357 (25h0m8.921503966s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.267543450 (25h0m9.161252059s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.337643945 (25h0m9.231352554s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.380001967 (25h0m9.273710576s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.850164228 (25h0m9.743872837s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.183873887 (25h0m10.077582496s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361354387 (25h0m10.255063026s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361383082 (25h0m10.255091691s) [seg_pos={0 0}, upload=21929]: Satisfied cohort requirements after 13 pieces Jan 16 16:56:13.361615564 (25h0m10.255324173s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361624061 (25h0m10.25533267s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361904004 (25h0m10.255612623s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: read tcp 188.166.225.170:42244->145.79.227.38:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 16 16:56:13.361925344 (25h0m10.255633963s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361971022 (25h0m10.255679631s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361978977 (25h0m10.255687596s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.362041817 (25h0m10.255750426s) [seg_pos={0 0}, upload=21929]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 16 16:56:13.362197272 (25h0m10.255905881s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 16 16:56:13.362205868 (25h0m10.255914477s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 16 16:56:13.965177888 (25h0m10.858886497s) [upload=21929]: Done uploading segments... Jan 16 16:56:13.965204158 (25h0m10.858912767s) [upload=21929]: Upload finished. err: <nil> Duration: 5.758793093s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 14.930880498s trace: https://jaeger.dev.storj.io/trace/70c2b0731de5fb9c FAIL (2026-01-16 17:28:20 UTC) orbiter warning: context deadline exceeded Duration: 14.930909183s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 53 out of 60 is failed (threshold=6) [XX.X..XXXXXXX.XXXXXXXXXXXXXX.XXXXXXXX.XXXXXXXXXXXXXXXXX.XXXX] duration: 5.001502447s trace: https://jaeger.dev.storj.io/trace/50299d38803028f2 FAIL (2026-01-16 17:28:16 UTC) orbiter warning: context deadline exceeded Duration: 5.001513168s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 25.29588469s trace: https://jaeger.dev.storj.io/trace/70c2b0731de5fb9c FAIL (2026-01-16 17:27:54 UTC) orbiter warning: context deadline exceeded Duration: 25.295909566s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................XXX...........X..................] duration: 18.741589772s trace: https://jaeger.dev.storj.io/trace/4a055e3bcb21e3db FAIL (2026-01-16 17:27:46 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 18.741654125s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 53 out of 60 is failed (threshold=6) [XXX.X..XXXXXXX.XXXXXXXXXXXXXX.XXXXXXXX.XXXXXXXXXXXXXXXXX.XXX] duration: 5.000762981s trace: https://jaeger.dev.storj.io/trace/1f2d40e598b74e42 FAIL (2026-01-16 17:27:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000785815s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 5.786312653s trace: https://jaeger.dev.storj.io/trace/ff1eaafc0d95989 FAIL (2026-01-16 17:27:25 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 16 16:56:06.586316499 (25h0m3.480025108s) [upload=21929]: Starting upload Jan 16 16:56:06.586371244 (25h0m3.480079853s) [upload=21929]: Done starting upload Jan 16 16:56:06.586529134 (25h0m3.480237763s) [upload=21929]: Uploading segments... Jan 16 16:56:06.586687636 (25h0m3.480396275s) [upload=21929]: Got next segment. Inline: false Jan 16 16:56:06.586750094 (25h0m3.480458723s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 16 16:56:06.586757138 (25h0m3.480465767s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 16 16:56:08.120364931 (25h0m5.01407354s) [seg_pos={0 0}, upload=21929]: Begin upload segment... Jan 16 16:56:08.120413474 (25h0m5.014122083s) [seg_pos={0 0}, upload=21929]: Done begin upload segment. Jan 16 16:56:08.120440615 (25h0m5.014149224s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.120799800 (25h0m5.014508419s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.121695454 (25h0m5.015404084s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.122317739 (25h0m5.016026349s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123161636 (25h0m5.016870245s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123395732 (25h0m5.017104351s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123848494 (25h0m5.017557113s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123868092 (25h0m5.017576731s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124249808 (25h0m5.017958417s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124389284 (25h0m5.018097893s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124619222 (25h0m5.018327831s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124999566 (25h0m5.018708175s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125202182 (25h0m5.018910801s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125576805 (25h0m5.019285414s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125952470 (25h0m5.019661079s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.126492900 (25h0m5.020201509s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:10.730578935 (25h0m7.624287544s) [upload=21929]: Next returned nil segment Jan 16 16:56:10.730594134 (25h0m7.624302753s) [upload=21929]: Waiting for error group managing segments... Jan 16 16:56:11.329683279 (25h0m8.223391888s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.338893335 (25h0m8.232601944s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400223651 (25h0m8.29393226s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400340974 (25h0m8.294049583s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.445450294 (25h0m8.339158904s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.964879212 (25h0m8.858587821s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.027795357 (25h0m8.921503966s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.267543450 (25h0m9.161252059s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.337643945 (25h0m9.231352554s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.380001967 (25h0m9.273710576s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.850164228 (25h0m9.743872837s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.183873887 (25h0m10.077582496s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361354387 (25h0m10.255063026s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361383082 (25h0m10.255091691s) [seg_pos={0 0}, upload=21929]: Satisfied cohort requirements after 13 pieces Jan 16 16:56:13.361615564 (25h0m10.255324173s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361624061 (25h0m10.25533267s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361904004 (25h0m10.255612623s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: read tcp 188.166.225.170:42244->145.79.227.38:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 16 16:56:13.361925344 (25h0m10.255633963s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361971022 (25h0m10.255679631s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361978977 (25h0m10.255687596s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.362041817 (25h0m10.255750426s) [seg_pos={0 0}, upload=21929]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 16 16:56:13.362197272 (25h0m10.255905881s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 16 16:56:13.362205868 (25h0m10.255914477s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 16 16:56:13.965177888 (25h0m10.858886497s) [upload=21929]: Done uploading segments... Jan 16 16:56:13.965204158 (25h0m10.858912767s) [upload=21929]: Upload finished. err: <nil> Duration: 5.7863422s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................XXX...........X.................] duration: 19.210534493s trace: https://jaeger.dev.storj.io/trace/4a055e3bcb21e3db FAIL (2026-01-16 17:27:15 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 19.210549312s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 53 out of 60 is failed (threshold=6) [XXXX.X..XXXXXXX.XXXXXXXXXXXXXX.XXXXXXXX.XXXXXXXXXXXXXXXXX.XX] duration: 5.00156582s trace: https://jaeger.dev.storj.io/trace/3d1e1bb6e7ee00b2 FAIL (2026-01-16 17:27:15 UTC) orbiter warning: context deadline exceeded Duration: 5.001586569s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 7.163844557s trace: https://jaeger.dev.storj.io/trace/70c2b0731de5fb9c FAIL (2026-01-16 17:27:09 UTC) orbiter warning: context deadline exceeded Duration: 7.163883622s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................XXX...........X................] duration: 18.120068506s trace: https://jaeger.dev.storj.io/trace/4a055e3bcb21e3db FAIL (2026-01-16 17:26:44 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 18.120097381s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 23.971905126s trace: https://jaeger.dev.storj.io/trace/70c2b0731de5fb9c FAIL (2026-01-16 17:26:24 UTC) orbiter warning: context deadline exceeded Duration: 23.971927999s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................XX........................................] duration: 5.745205341s trace: https://jaeger.dev.storj.io/trace/ff1eaafc0d95989 FAIL (2026-01-16 17:26:12 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 16 16:56:06.586316499 (25h0m3.480025108s) [upload=21929]: Starting upload Jan 16 16:56:06.586371244 (25h0m3.480079853s) [upload=21929]: Done starting upload Jan 16 16:56:06.586529134 (25h0m3.480237763s) [upload=21929]: Uploading segments... Jan 16 16:56:06.586687636 (25h0m3.480396275s) [upload=21929]: Got next segment. Inline: false Jan 16 16:56:06.586750094 (25h0m3.480458723s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 16 16:56:06.586757138 (25h0m3.480465767s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 16 16:56:08.120364931 (25h0m5.01407354s) [seg_pos={0 0}, upload=21929]: Begin upload segment... Jan 16 16:56:08.120413474 (25h0m5.014122083s) [seg_pos={0 0}, upload=21929]: Done begin upload segment. Jan 16 16:56:08.120440615 (25h0m5.014149224s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.120799800 (25h0m5.014508419s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.121695454 (25h0m5.015404084s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.122317739 (25h0m5.016026349s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123161636 (25h0m5.016870245s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123395732 (25h0m5.017104351s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123848494 (25h0m5.017557113s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123868092 (25h0m5.017576731s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124249808 (25h0m5.017958417s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124389284 (25h0m5.018097893s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124619222 (25h0m5.018327831s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124999566 (25h0m5.018708175s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125202182 (25h0m5.018910801s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125576805 (25h0m5.019285414s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125952470 (25h0m5.019661079s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.126492900 (25h0m5.020201509s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:10.730578935 (25h0m7.624287544s) [upload=21929]: Next returned nil segment Jan 16 16:56:10.730594134 (25h0m7.624302753s) [upload=21929]: Waiting for error group managing segments... Jan 16 16:56:11.329683279 (25h0m8.223391888s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.338893335 (25h0m8.232601944s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400223651 (25h0m8.29393226s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400340974 (25h0m8.294049583s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.445450294 (25h0m8.339158904s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.964879212 (25h0m8.858587821s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.027795357 (25h0m8.921503966s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.267543450 (25h0m9.161252059s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.337643945 (25h0m9.231352554s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.380001967 (25h0m9.273710576s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.850164228 (25h0m9.743872837s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.183873887 (25h0m10.077582496s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361354387 (25h0m10.255063026s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361383082 (25h0m10.255091691s) [seg_pos={0 0}, upload=21929]: Satisfied cohort requirements after 13 pieces Jan 16 16:56:13.361615564 (25h0m10.255324173s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361624061 (25h0m10.25533267s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361904004 (25h0m10.255612623s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: read tcp 188.166.225.170:42244->145.79.227.38:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 16 16:56:13.361925344 (25h0m10.255633963s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361971022 (25h0m10.255679631s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361978977 (25h0m10.255687596s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.362041817 (25h0m10.255750426s) [seg_pos={0 0}, upload=21929]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 16 16:56:13.362197272 (25h0m10.255905881s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 16 16:56:13.362205868 (25h0m10.255914477s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 16 16:56:13.965177888 (25h0m10.858886497s) [upload=21929]: Done uploading segments... Jan 16 16:56:13.965204158 (25h0m10.858912767s) [upload=21929]: Upload finished. err: <nil> Duration: 5.745225138s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................XXX...........X...............] duration: 20.639697148s trace: https://jaeger.dev.storj.io/trace/4a055e3bcb21e3db FAIL (2026-01-16 17:26:08 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 20.639722717s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 53 out of 60 is failed (threshold=6) [XXXXX.X..XXXXXXX.XXXXXXXXXXXXXX.XXXXXXXX.XXXXXXXXXXXXXXXXX.X] duration: 5.000719257s trace: https://jaeger.dev.storj.io/trace/45c4deae89243900 FAIL (2026-01-16 17:26:04 UTC) orbiter warning: context deadline exceeded Duration: 5.000740929s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 18.952329123s trace: https://jaeger.dev.storj.io/trace/70c2b0731de5fb9c FAIL (2026-01-16 17:25:57 UTC) orbiter warning: context deadline exceeded Duration: 18.952358128s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 5.742778753s trace: https://jaeger.dev.storj.io/trace/ff1eaafc0d95989 FAIL (2026-01-16 17:25:34 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 16 16:56:06.586316499 (25h0m3.480025108s) [upload=21929]: Starting upload Jan 16 16:56:06.586371244 (25h0m3.480079853s) [upload=21929]: Done starting upload Jan 16 16:56:06.586529134 (25h0m3.480237763s) [upload=21929]: Uploading segments... Jan 16 16:56:06.586687636 (25h0m3.480396275s) [upload=21929]: Got next segment. Inline: false Jan 16 16:56:06.586750094 (25h0m3.480458723s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 16 16:56:06.586757138 (25h0m3.480465767s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 16 16:56:08.120364931 (25h0m5.01407354s) [seg_pos={0 0}, upload=21929]: Begin upload segment... Jan 16 16:56:08.120413474 (25h0m5.014122083s) [seg_pos={0 0}, upload=21929]: Done begin upload segment. Jan 16 16:56:08.120440615 (25h0m5.014149224s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.120799800 (25h0m5.014508419s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.121695454 (25h0m5.015404084s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.122317739 (25h0m5.016026349s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123161636 (25h0m5.016870245s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123395732 (25h0m5.017104351s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123848494 (25h0m5.017557113s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.123868092 (25h0m5.017576731s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124249808 (25h0m5.017958417s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124389284 (25h0m5.018097893s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124619222 (25h0m5.018327831s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.124999566 (25h0m5.018708175s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125202182 (25h0m5.018910801s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125576805 (25h0m5.019285414s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.125952470 (25h0m5.019661079s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:08.126492900 (25h0m5.020201509s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Uploading piece... Jan 16 16:56:10.730578935 (25h0m7.624287544s) [upload=21929]: Next returned nil segment Jan 16 16:56:10.730594134 (25h0m7.624302753s) [upload=21929]: Waiting for error group managing segments... Jan 16 16:56:11.329683279 (25h0m8.223391888s) [noise=true , address=145.79.227.15:4873 , piece_id=EVJGQ5JNJI37QIPQDJPFOPMT3JAHPBZXQG7DBRUZXJMKXAIADWSA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.338893335 (25h0m8.232601944s) [noise=true , address=45.91.116.14:4190 , piece_id=ZBC7C5E5V7B4VAGI2JHD7T4E6BGVV74YNNAROZYBITS2N23BJDWA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400223651 (25h0m8.29393226s) [noise=true , address=145.79.227.33:4870 , piece_id=7I3KU7EKC2MHBJ6TOGO5ZBMJSPYU4E26D5373M2R53IZAZJLWVOA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.400340974 (25h0m8.294049583s) [noise=true , address=145.79.227.2:4115 , piece_id=YMME5WUV7XGQZOE2I5H7UWQ7TSTP2LGV7D4AZO4T62XOWBE4ZEKA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.445450294 (25h0m8.339158904s) [noise=true , address=145.79.227.30:4762 , piece_id=YLBYKHJNJ2J3YYVYHRWEZTFLXZVL5WCKCBWXM526XF4KKCYA4RAQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:11.964879212 (25h0m8.858587821s) [noise=true , address=145.79.227.3:4151 , piece_id=QVFK6RJITYJIW6ZJRUYFU3P2NEMMVANA36O6WXPS7Z7FNFSYQWUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.027795357 (25h0m8.921503966s) [noise=true , address=145.79.227.49:4339 , piece_id=NWDULUWHMV4PIMWMNO5XYXSAGVLNPLSIPYSDA32OZB2MBGBYPEUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.267543450 (25h0m9.161252059s) [noise=true , address=145.79.227.28:4727 , piece_id=U5CDDUXQZ2BB7SPFAIWQXRGALJQTOWFNWGJLU3CGXKBUHYIQBIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.337643945 (25h0m9.231352554s) [noise=true , address=145.79.227.10:4401 , piece_id=C2X52SPGSNM2CB6TFGBCT6HO3MXSEMX3CCZQASDNFGWIEW2EGIGQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.380001967 (25h0m9.273710576s) [noise=true , address=145.79.227.32:4835 , piece_id=6PP7J34OCTKYEDEFQJWDUHO5H4Z6VG7LNG3SL7WLITSKMX43HAYQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:12.850164228 (25h0m9.743872837s) [noise=true , address=77.90.31.13:4627 , piece_id=TJFFMEVYN7M52FH5UP3WNAGC7HWF5GSEUPQJZ5KJWTJZWKZNAWXQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.183873887 (25h0m10.077582496s) [noise=true , address=77.90.31.19:4842 , piece_id=3VJ3J4JQPSJEVFSNSYDVHRFZMMCBMNZRBYEGJOOU4Q4HFEDDOKHQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361354387 (25h0m10.255063026s) [noise=true , address=45.91.116.4:4937 , piece_id=MJHRB2I4KWRJJ3MHGW657D53JFDARTHOBRUEHQZEUA4DRFVK7ZUA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: <nil> Jan 16 16:56:13.361383082 (25h0m10.255091691s) [seg_pos={0 0}, upload=21929]: Satisfied cohort requirements after 13 pieces Jan 16 16:56:13.361615564 (25h0m10.255324173s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361624061 (25h0m10.25533267s) [noise=true , address=145.79.227.34:4906 , piece_id=G2GHF4S5NVJP5R43EN6NDEJ4FCPCLSL5K3TO4UADJ6ZANME54KOQ, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361904004 (25h0m10.255612623s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8): protocol: read tcp 188.166.225.170:42244->145.79.227.38:4977: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 16 16:56:13.361925344 (25h0m10.255633963s) [noise=true , address=145.79.227.38:4977 , piece_id=JXV6MGOVSAA2URSIF2KFFQJJEGWQWFVT65PWVDW5MEH5ITSDYFAA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.361971022 (25h0m10.255679631s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: expected piece hash; context canceled; EOF Jan 16 16:56:13.361978977 (25h0m10.255687596s) [noise=true , address=45.91.116.5:4300 , piece_id=M2F4W7IGAEKEXJXZRENON55A2E7DNSCZTPKWYJFY3NKNTBIWG2YA, seg_pos={0 0}, upload=21929]: Long tail cancellation due to optimal threshold Jan 16 16:56:13.362041817 (25h0m10.255750426s) [seg_pos={0 0}, upload=21929]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 16 16:56:13.362197272 (25h0m10.255905881s) [upload=21929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 16 16:56:13.362205868 (25h0m10.255914477s) [upload=21929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 16 16:56:13.965177888 (25h0m10.858886497s) [upload=21929]: Done uploading segments... Jan 16 16:56:13.965204158 (25h0m10.858912767s) [upload=21929]: Upload finished. err: <nil> Duration: 5.74279838s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 53 out of 60 is failed (threshold=6) [XXXXXX.X..XXXXXXX.XXXXXXXXXXXXXX.XXXXXXXX.XXXXXXXXXXXXXXXXX.] duration: 3.639266571s trace: https://jaeger.dev.storj.io/trace/51fbcd5c3ae44c5b FAIL (2026-01-16 17:25:26 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.639292651s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................XXX...........X..............] duration: 17.890919637s trace: https://jaeger.dev.storj.io/trace/4a055e3bcb21e3db FAIL (2026-01-16 17:25:10 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 17.890948002s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)