Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
6.79720057s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:27:45 UTC)
orbiter warning: context deadline exceeded
Duration: 6.797220357s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
6.78150337s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:27:40 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.781521835s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
3.967898237s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:27:11 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.967919418s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
6.78269353s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:26:55 UTC)
orbiter warning: context deadline exceeded
Duration: 6.782712146s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
10.701943841s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:26:43 UTC)
orbiter warning: context deadline exceeded
Duration: 10.701978266s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
8.481197716s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:26:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 8.481217704s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
6.478206081s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:26:00 UTC)
orbiter warning: context deadline exceeded
Duration: 6.47823116s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
4.390323857s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:25:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.390344087s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
4.004039477s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:24:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.004065227s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
6.559289421s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:24:51 UTC)
orbiter warning: context deadline exceeded
Duration: 6.559311024s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
6.453811778s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:24:23 UTC)
orbiter warning: context deadline exceeded
Duration: 6.453830113s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
4.456326185s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:24:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.456404273s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
6.556114938s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:23:50 UTC)
orbiter warning: context deadline exceeded
Duration: 6.55612633s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
4.179895057s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:23:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.179935675s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
7.703556317s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:22:59 UTC)
orbiter warning: context deadline exceeded
Duration: 7.703576375s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
4.136174741s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:22:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.13618897s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
6.53628392s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:22:01 UTC)
orbiter warning: context deadline exceeded
Duration: 6.536320259s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
4.082196509s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:21:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.082215376s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
6.530445269s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:21:20 UTC)
orbiter warning: context deadline exceeded
Duration: 6.530465539s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
7.922815458s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:21:17 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 7.922840878s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
6.698386093s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:20:52 UTC)
orbiter warning: context deadline exceeded
Duration: 6.698408295s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [X............................................X..............] duration:
4.278727607s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:20:37 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.278754157s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
6.385027471s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:20:21 UTC)
orbiter warning: context deadline exceeded
Duration: 6.385054502s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.X............................................X.............] duration:
4.261111995s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:19:46 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.261149306s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
8.255211236s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:19:28 UTC)
orbiter warning: context deadline exceeded
Duration: 8.25522897s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..X............................................X............] duration:
4.222979424s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:18:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.222991237s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
6.782992569s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:18:39 UTC)
orbiter warning: context deadline exceeded
Duration: 6.783011546s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...X............................................X...........] duration:
4.135862087s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:18:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.135882015s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
6.746413466s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:18:02 UTC)
orbiter warning: context deadline exceeded
Duration: 6.746453683s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
6.618484225s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:17:30 UTC)
orbiter warning: context deadline exceeded
Duration: 6.618504644s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [....X............................................X..........] duration:
5.556226403s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:17:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.556241282s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
6.658250612s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:16:48 UTC)
orbiter warning: context deadline exceeded
Duration: 6.658277894s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.....X............................................X.........] duration:
4.282861965s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:16:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.282892433s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
6.720450224s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:16:31 UTC)
orbiter warning: context deadline exceeded
Duration: 6.720460263s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.001089782s trace:
https://jaeger.dev.storj.io/trace/3dd35135526dbf86
FAIL (2026-05-25 10:16:24 UTC)
orbiter warning: context deadline exceeded
Duration: 1m0.001108978s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [......X............................................X........] duration:
4.416471961s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:15:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.416512389s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.......X............................................X.......] duration:
3.884018087s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:14:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.884041182s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [........X............................................X......] duration:
4.020256264s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:13:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.020277485s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.........X............................................X.....] duration:
3.926970506s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:12:52 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.927022306s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..........X............................................X....] duration:
3.967991257s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:11:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.968014011s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...........X............................................X...] duration:
4.494460712s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:11:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.494484347s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [............X............................................X..] duration:
4.235244818s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:10:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.235270476s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.............X............................................X.] duration:
3.962529824s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:09:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.962541939s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..............X............................................X] duration:
1m17.057497709s trace:
https://jaeger.dev.storj.io/trace/1034ec799a20bbc
FAIL (2026-05-25 10:09:37 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
35372577s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852396903 (8h4m28.635737002s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852641450 (8h4m28.635981549s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.852876398 (8h4m28.636216487s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853119302 (8h4m28.636459401s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853452037 (8h4m28.636792136s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853758661 (8h4m28.637098761s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.853931502 (8h4m28.637271601s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854257905 (8h4m28.637598004s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854608202 (8h4m28.637948301s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.854945245 (8h4m28.638285354s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855326673 (8h4m28.638666772s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.855651373 (8h4m28.638991472s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856007261 (8h4m28.639347351s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856391575 (8h4m28.639731664s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:42.856730622 (8h4m28.640070721s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Uploading piece...
May 25 10:08:43.927482841 (8h4m29.71082293s) [upload=7753]: Next returned nil segment
May 25 10:08:43.927494334 (8h4m29.710834433s) [upload=7753]: Waiting for error group managing segments...
May 25 10:08:44.208156829 (8h4m29.991496928s) [noise=true , address=212.102.52.213:4334 , piece_id=ZI7LMSC2SXFQM63533ZM2YU53CJVFEUZCHPEM3IJN677SHBSFBGA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.209504721 (8h4m29.99284482s) [noise=true , address=212.102.52.214:4366 , piece_id=SKBELBNPAEF3KGXLA22OPQU3JH7IQ4YZTVHYDMGDIHDZQJV7OGDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.210864685 (8h4m29.994204785s) [noise=true , address=212.102.52.214:4358 , piece_id=7T5PLGN2CLOXMVMO3X5L2NRZXZWYJRNIZ7RRXXQGRCSMA63MDUBA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.215417934 (8h4m29.998758043s) [noise=true , address=212.102.52.214:4361 , piece_id=FM67X2ZRRG5PXSRRQ243TVVQNGBQ6RQKKYXQB4F44GBM6BMZZWVA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.218804034 (8h4m30.002144123s) [noise=true , address=212.102.52.215:4386 , piece_id=NOMTX5PRRXBSI4H7GDKZ5FX7EHD3GCBPAN4AASCERCIMW3MAOZAA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219271346 (8h4m30.002611445s) [noise=true , address=212.102.52.215:4387 , piece_id=FXPN5MLL3LWZZCQT4YLQLEBYBWL33E2BXGQTICKW26AVQHFRF62A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219342231 (8h4m30.00268233s) [noise=true , address=212.102.52.216:4412 , piece_id=NM27G5B3FXWVBAOFPY5HYWKIUW2IW7J6PSH4FIZN24NAEN4ZFS2Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.219683082 (8h4m30.003023181s) [noise=true , address=212.102.52.213:4327 , piece_id=GZ7XZDEZPK2WXSG4REGKCP4IOEQ5IX3WITRWUROTL275EQBVG2QA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.221411550 (8h4m30.004751649s) [noise=true , address=212.102.52.217:4432 , piece_id=AYI4MSDB5YFLHAMWSEFMFFSX2HRVL5YUP3BSDFDS7PXUJN7NZUSQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.226560133 (8h4m30.009900222s) [noise=true , address=212.102.52.217:4418 , piece_id=B4S3CEBD5AGGWMJA2MUTZM7EZWEJNJKUIT65IYVXE6AKMAXW7EMA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.228266380 (8h4m30.011606469s) [noise=true , address=212.102.52.215:4385 , piece_id=PNSKLE65V2UZBECR5RTKIJSM2S3C6VDX3IAE4NFK6CUWXAPJGWSA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.235448804 (8h4m30.018788903s) [noise=true , address=212.102.52.217:4427 , piece_id=AFV4R35RCIYQSFUWTBAAVTVRJ7AKYODM2MKMOJ72RPRPAEGWUHYQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.273458310 (8h4m30.056798409s) [noise=true , address=212.102.52.216:4393 , piece_id=FMGYLRSBPUAYWHG7MB6E2RHYDFNH2VBOLF4SLRUVBCYAMGULE6GQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.274640074 (8h4m30.057980173s) [noise=true , address=212.102.52.216:4405 , piece_id=2M2VPAGHYKBUSQCVE7OPNSXLFBGFWBWY2HFPYDUNWDCQ76NRI3XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.279448971 (8h4m30.06278907s) [noise=true , address=212.102.52.216:4400 , piece_id=2Y5WLNNRJF36F2CTPTO7FQ5A2OBPEKRBRD5LEAK6WAZJRZPTMQLA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284933956 (8h4m30.068274055s) [noise=true , address=212.102.52.217:4423 , piece_id=LUTBVKXF3Q4QBYKXCL4YNOS7UE7X4N35ZM62JJDCIBEMEAREMMFQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil>
May 25 10:08:44.284947773 (8h4m30.068287872s) [seg_pos={0 0}, upload=7753]: Satisfied cohort requirements after 16 pieces
May 25 10:08:44.285158565 (8h4m30.068498654s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16QRjPkVjsvSczuSVS3M22YoZGSHqcDrMECQmBrwv55v43bHkG): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285173503 (8h4m30.068513602s) [noise=true , address=212.102.52.215:4379 , piece_id=IVFCHESTQXW3GFH77IPY7QNNRFK73PP4ORV4CE7BTKMEKYUQXXMQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285195565 (8h4m30.068535654s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285203921 (8h4m30.06854402s) [noise=true , address=212.102.52.213:4329 , piece_id=6YOQFDDY4J4SPCFEI3GYECTA3OYZ5QD225SDW7VSFRTEVPUMG7XQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285393914 (8h4m30.068734013s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cwAu1rBbxVBvbgfqzFyWSjF8t6Si8Z32BLiHuVzvkAGbLb7TR): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285400787 (8h4m30.068740886s) [noise=true , address=212.102.52.214:4362 , piece_id=IV7EDHH3WBDUYFZST7C46Z2KDJTXOPHDOHRGQFNZJZNYOET6AXRQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285531656 (8h4m30.068871755s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15oktwdg3Zq7QEGkDpv7PUCi9WXZqj7HsGRyzbvp31uWhhQrKC): protocol: expected piece hash; context canceled; EOF
May 25 10:08:44.285537628 (8h4m30.068877727s) [noise=true , address=212.102.52.213:4332 , piece_id=P2RTZJC2UBUDCOXKNNKX4EKUQMHSJFJOFLAUUUFS75KHUR43XKVQ, seg_pos={0 0}, upload=7753]: Long tail cancellation due to optimal threshold
May 25 10:08:44.285562736 (8h4m30.068902835s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 10:08:44.285617260 (8h4m30.068957359s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 10:08:44.285620696 (8h4m30.068960795s) [upload=7753]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 10:09:37.870454601 (8h5m23.6537947s) [upload=7753]: Done uploading segments...
May 25 10:09:37.870520758 (8h5m23.653860857s) [upload=7753]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1m17.057517327s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
9.343892776s trace:
https://jaeger.dev.storj.io/trace/4eb585f30f70f6a
FAIL (2026-05-25 10:07:20 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
e=true , address=212.102.52.216:4399 , piece_id=4F2RKDATBD4ARRQ2W7WJLD6AKHZQEWSQN3BYS2XXS4CTWUCH2PQA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.830098022 (7h28m29.613438131s) [noise=true , address=212.102.52.216:4414 , piece_id=WNAXREPANUDI4NEYQBDUL75SY4Z5MLPGE2ORIOJRQTPVZX2CWAQQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.830837252 (7h28m29.614177351s) [noise=true , address=212.102.52.217:4419 , piece_id=SJW7AM4LK4JSQAQQK2GMSRS3LHSI5EZZBAAIWKYTM7VADEW4MRLA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.831733071 (7h28m29.61507317s) [noise=true , address=212.102.52.215:4385 , piece_id=5TDXQEJVMZLZK3OH53T36LHHVPAMZD2MF6E4YHJMUCFOKU7TMQKQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.832187388 (7h28m29.615527487s) [noise=true , address=212.102.52.213:4343 , piece_id=JJHIT42VPZI6BWRJMC5DBNW32VEIGAN3NWVYDZUN7NN7JMJSXIMA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.832692371 (7h28m29.61603246s) [noise=true , address=212.102.52.213:4324 , piece_id=YMEXBDFUX7FDPUGI4YMKDA3VY2ONIGSFH7QPMTLHC2FNCFVX4EGA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.833491967 (7h28m29.616832066s) [noise=true , address=212.102.52.213:4331 , piece_id=O7TPMVI2SUBJX4PUU7XI4EXVP6B772C6OVIIKGOH4H7BKBV7WFPA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.833985479 (7h28m29.617325578s) [noise=true , address=212.102.52.217:4440 , piece_id=EJGXBWWFIQDR7WVUF3ROPORIV4LFKNYQAHZGPT4GQT4M2ASB2VTA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.836004521 (7h28m29.61934463s) [noise=true , address=212.102.52.217:4422 , piece_id=VVXCZLY77G7RFK25ECZ2UJPAOGWLSELFXB53PIHNIKKCN5TSAB6A, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.838800066 (7h28m29.622140165s) [noise=true , address=212.102.52.217:4430 , piece_id=PN57VWOAPH7U2PPCV5YVQUYPACWLQIDZOP6BRCIPPSCPB23X5ZHQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.840366935 (7h28m29.623707024s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.842006614 (7h28m29.625346713s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.843201753 (7h28m29.626541853s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.845076260 (7h28m29.628416359s) [noise=true , address=212.102.52.214:4364 , piece_id=6ZFNYSVRTKGVJVZIAEYJM5LQUHTVJO4SO5JYGWAISF4TBT4RLTHQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.845505149 (7h28m29.628845258s) [noise=true , address=212.102.52.215:4373 , piece_id=AKUFHKJGCCZ4JB4WGCSJCBLUUI5PAO4F5ZROGHTRUKH7FRRLTOVQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.847289253 (7h28m29.630629352s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:44.897115628 (7h28m30.680455727s) [upload=7184]: Next returned nil segment
May 25 09:32:44.897126990 (7h28m30.680467089s) [upload=7184]: Waiting for error group managing segments...
May 25 09:32:45.181306033 (7h28m30.964646133s) [noise=true , address=212.102.52.216:4414 , piece_id=WNAXREPANUDI4NEYQBDUL75SY4Z5MLPGE2ORIOJRQTPVZX2CWAQQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.182774646 (7h28m30.966114745s) [noise=true , address=212.102.52.216:4408 , piece_id=V2UU3QK5OYPDBL23FUVYCNUTPE6S3BQZL3XWCVAZXZPSECJTRE3Q, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.182945902 (7h28m30.966286001s) [noise=true , address=212.102.52.216:4400 , piece_id=5PJ7CFDLRQE3YPUOATP7O7POFP4NGTRFZKYXVX4KLZTPZULLZ4XA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.185420764 (7h28m30.968760863s) [noise=true , address=212.102.52.216:4399 , piece_id=4F2RKDATBD4ARRQ2W7WJLD6AKHZQEWSQN3BYS2XXS4CTWUCH2PQA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.186002044 (7h28m30.969342143s) [noise=true , address=212.102.52.214:4367 , piece_id=6O5TT7K373VMSSNWNLUVAH7JJPK3Y2MHKQSWYNTULHT26BJJTU2A, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.187072957 (7h28m30.970413056s) [noise=true , address=212.102.52.213:4330 , piece_id=4FP2RPTIETYETJMKUT3XRK7GBKR45TWLNPGJ5ORQYHNTN6LY2DMQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.188495802 (7h28m30.971835901s) [noise=true , address=212.102.52.217:4430 , piece_id=PN57VWOAPH7U2PPCV5YVQUYPACWLQIDZOP6BRCIPPSCPB23X5ZHQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.195292711 (7h28m30.97863281s) [noise=true , address=212.102.52.213:4343 , piece_id=JJHIT42VPZI6BWRJMC5DBNW32VEIGAN3NWVYDZUN7NN7JMJSXIMA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.199852751 (7h28m30.98319285s) [noise=true , address=212.102.52.215:4385 , piece_id=5TDXQEJVMZLZK3OH53T36LHHVPAMZD2MF6E4YHJMUCFOKU7TMQKQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.200159136 (7h28m30.983499225s) [noise=true , address=212.102.52.217:4440 , piece_id=EJGXBWWFIQDR7WVUF3ROPORIV4LFKNYQAHZGPT4GQT4M2ASB2VTA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.203990647 (7h28m30.987330746s) [noise=true , address=212.102.52.213:4331 , piece_id=O7TPMVI2SUBJX4PUU7XI4EXVP6B772C6OVIIKGOH4H7BKBV7WFPA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.204341486 (7h28m30.987681585s) [noise=true , address=212.102.52.215:4373 , piece_id=AKUFHKJGCCZ4JB4WGCSJCBLUUI5PAO4F5ZROGHTRUKH7FRRLTOVQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.207397948 (7h28m30.990738047s) [noise=true , address=212.102.52.217:4422 , piece_id=VVXCZLY77G7RFK25ECZ2UJPAOGWLSELFXB53PIHNIKKCN5TSAB6A, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.209282203 (7h28m30.992622302s) [noise=true , address=212.102.52.213:4324 , piece_id=YMEXBDFUX7FDPUGI4YMKDA3VY2ONIGSFH7QPMTLHC2FNCFVX4EGA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.210214793 (7h28m30.993554882s) [noise=true , address=212.102.52.214:4364 , piece_id=6ZFNYSVRTKGVJVZIAEYJM5LQUHTVJO4SO5JYGWAISF4TBT4RLTHQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.211913804 (7h28m30.995253903s) [noise=true , address=212.102.52.217:4419 , piece_id=SJW7AM4LK4JSQAQQK2GMSRS3LHSI5EZZBAAIWKYTM7VADEW4MRLA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.211924885 (7h28m30.995264984s) [seg_pos={0 0}, upload=7184]: Satisfied cohort requirements after 16 pieces
May 25 09:32:45.212150666 (7h28m30.995490765s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ku2quD8AHh79sZcysV979DNjGBVGAazWZM7P5vPeckaKfoBL5): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212161617 (7h28m30.995501716s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212170323 (7h28m30.995510422s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212177678 (7h28m30.995517777s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212287968 (7h28m30.995628067s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J6FD4Y3i9TV7hgSi3QNR8Bf27t9oenvhecVTgy4cMySGFw6Go): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212292587 (7h28m30.995632676s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212433165 (7h28m30.995773264s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212443646 (7h28m30.995783735s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212461499 (7h28m30.995801598s) [seg_pos={0 0}, upload=7184]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 09:32:45.212537665 (7h28m30.995877764s) [upload=7184]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 09:32:45.212542644 (7h28m30.995882743s) [upload=7184]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 09:32:53.332110792 (7h28m39.115450931s) [upload=7184]: Done uploading segments...
May 25 09:32:53.332134266 (7h28m39.115474395s) [upload=7184]: Upload finished. err: <nil>
Duration: 9.343914128s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
4.227220441s trace:
https://jaeger.dev.storj.io/trace/4eb585f30f70f6a
FAIL (2026-05-25 10:06:37 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
e=true , address=212.102.52.216:4399 , piece_id=4F2RKDATBD4ARRQ2W7WJLD6AKHZQEWSQN3BYS2XXS4CTWUCH2PQA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.830098022 (7h28m29.613438131s) [noise=true , address=212.102.52.216:4414 , piece_id=WNAXREPANUDI4NEYQBDUL75SY4Z5MLPGE2ORIOJRQTPVZX2CWAQQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.830837252 (7h28m29.614177351s) [noise=true , address=212.102.52.217:4419 , piece_id=SJW7AM4LK4JSQAQQK2GMSRS3LHSI5EZZBAAIWKYTM7VADEW4MRLA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.831733071 (7h28m29.61507317s) [noise=true , address=212.102.52.215:4385 , piece_id=5TDXQEJVMZLZK3OH53T36LHHVPAMZD2MF6E4YHJMUCFOKU7TMQKQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.832187388 (7h28m29.615527487s) [noise=true , address=212.102.52.213:4343 , piece_id=JJHIT42VPZI6BWRJMC5DBNW32VEIGAN3NWVYDZUN7NN7JMJSXIMA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.832692371 (7h28m29.61603246s) [noise=true , address=212.102.52.213:4324 , piece_id=YMEXBDFUX7FDPUGI4YMKDA3VY2ONIGSFH7QPMTLHC2FNCFVX4EGA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.833491967 (7h28m29.616832066s) [noise=true , address=212.102.52.213:4331 , piece_id=O7TPMVI2SUBJX4PUU7XI4EXVP6B772C6OVIIKGOH4H7BKBV7WFPA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.833985479 (7h28m29.617325578s) [noise=true , address=212.102.52.217:4440 , piece_id=EJGXBWWFIQDR7WVUF3ROPORIV4LFKNYQAHZGPT4GQT4M2ASB2VTA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.836004521 (7h28m29.61934463s) [noise=true , address=212.102.52.217:4422 , piece_id=VVXCZLY77G7RFK25ECZ2UJPAOGWLSELFXB53PIHNIKKCN5TSAB6A, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.838800066 (7h28m29.622140165s) [noise=true , address=212.102.52.217:4430 , piece_id=PN57VWOAPH7U2PPCV5YVQUYPACWLQIDZOP6BRCIPPSCPB23X5ZHQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.840366935 (7h28m29.623707024s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.842006614 (7h28m29.625346713s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.843201753 (7h28m29.626541853s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.845076260 (7h28m29.628416359s) [noise=true , address=212.102.52.214:4364 , piece_id=6ZFNYSVRTKGVJVZIAEYJM5LQUHTVJO4SO5JYGWAISF4TBT4RLTHQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.845505149 (7h28m29.628845258s) [noise=true , address=212.102.52.215:4373 , piece_id=AKUFHKJGCCZ4JB4WGCSJCBLUUI5PAO4F5ZROGHTRUKH7FRRLTOVQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.847289253 (7h28m29.630629352s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:44.897115628 (7h28m30.680455727s) [upload=7184]: Next returned nil segment
May 25 09:32:44.897126990 (7h28m30.680467089s) [upload=7184]: Waiting for error group managing segments...
May 25 09:32:45.181306033 (7h28m30.964646133s) [noise=true , address=212.102.52.216:4414 , piece_id=WNAXREPANUDI4NEYQBDUL75SY4Z5MLPGE2ORIOJRQTPVZX2CWAQQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.182774646 (7h28m30.966114745s) [noise=true , address=212.102.52.216:4408 , piece_id=V2UU3QK5OYPDBL23FUVYCNUTPE6S3BQZL3XWCVAZXZPSECJTRE3Q, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.182945902 (7h28m30.966286001s) [noise=true , address=212.102.52.216:4400 , piece_id=5PJ7CFDLRQE3YPUOATP7O7POFP4NGTRFZKYXVX4KLZTPZULLZ4XA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.185420764 (7h28m30.968760863s) [noise=true , address=212.102.52.216:4399 , piece_id=4F2RKDATBD4ARRQ2W7WJLD6AKHZQEWSQN3BYS2XXS4CTWUCH2PQA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.186002044 (7h28m30.969342143s) [noise=true , address=212.102.52.214:4367 , piece_id=6O5TT7K373VMSSNWNLUVAH7JJPK3Y2MHKQSWYNTULHT26BJJTU2A, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.187072957 (7h28m30.970413056s) [noise=true , address=212.102.52.213:4330 , piece_id=4FP2RPTIETYETJMKUT3XRK7GBKR45TWLNPGJ5ORQYHNTN6LY2DMQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.188495802 (7h28m30.971835901s) [noise=true , address=212.102.52.217:4430 , piece_id=PN57VWOAPH7U2PPCV5YVQUYPACWLQIDZOP6BRCIPPSCPB23X5ZHQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.195292711 (7h28m30.97863281s) [noise=true , address=212.102.52.213:4343 , piece_id=JJHIT42VPZI6BWRJMC5DBNW32VEIGAN3NWVYDZUN7NN7JMJSXIMA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.199852751 (7h28m30.98319285s) [noise=true , address=212.102.52.215:4385 , piece_id=5TDXQEJVMZLZK3OH53T36LHHVPAMZD2MF6E4YHJMUCFOKU7TMQKQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.200159136 (7h28m30.983499225s) [noise=true , address=212.102.52.217:4440 , piece_id=EJGXBWWFIQDR7WVUF3ROPORIV4LFKNYQAHZGPT4GQT4M2ASB2VTA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.203990647 (7h28m30.987330746s) [noise=true , address=212.102.52.213:4331 , piece_id=O7TPMVI2SUBJX4PUU7XI4EXVP6B772C6OVIIKGOH4H7BKBV7WFPA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.204341486 (7h28m30.987681585s) [noise=true , address=212.102.52.215:4373 , piece_id=AKUFHKJGCCZ4JB4WGCSJCBLUUI5PAO4F5ZROGHTRUKH7FRRLTOVQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.207397948 (7h28m30.990738047s) [noise=true , address=212.102.52.217:4422 , piece_id=VVXCZLY77G7RFK25ECZ2UJPAOGWLSELFXB53PIHNIKKCN5TSAB6A, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.209282203 (7h28m30.992622302s) [noise=true , address=212.102.52.213:4324 , piece_id=YMEXBDFUX7FDPUGI4YMKDA3VY2ONIGSFH7QPMTLHC2FNCFVX4EGA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.210214793 (7h28m30.993554882s) [noise=true , address=212.102.52.214:4364 , piece_id=6ZFNYSVRTKGVJVZIAEYJM5LQUHTVJO4SO5JYGWAISF4TBT4RLTHQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.211913804 (7h28m30.995253903s) [noise=true , address=212.102.52.217:4419 , piece_id=SJW7AM4LK4JSQAQQK2GMSRS3LHSI5EZZBAAIWKYTM7VADEW4MRLA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.211924885 (7h28m30.995264984s) [seg_pos={0 0}, upload=7184]: Satisfied cohort requirements after 16 pieces
May 25 09:32:45.212150666 (7h28m30.995490765s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ku2quD8AHh79sZcysV979DNjGBVGAazWZM7P5vPeckaKfoBL5): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212161617 (7h28m30.995501716s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212170323 (7h28m30.995510422s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212177678 (7h28m30.995517777s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212287968 (7h28m30.995628067s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J6FD4Y3i9TV7hgSi3QNR8Bf27t9oenvhecVTgy4cMySGFw6Go): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212292587 (7h28m30.995632676s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212433165 (7h28m30.995773264s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212443646 (7h28m30.995783735s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212461499 (7h28m30.995801598s) [seg_pos={0 0}, upload=7184]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 09:32:45.212537665 (7h28m30.995877764s) [upload=7184]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 09:32:45.212542644 (7h28m30.995882743s) [upload=7184]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 09:32:53.332110792 (7h28m39.115450931s) [upload=7184]: Done uploading segments...
May 25 09:32:53.332134266 (7h28m39.115474395s) [upload=7184]: Upload finished. err: <nil>
Duration: 4.227241742s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.60660708s trace:
https://jaeger.dev.storj.io/trace/4eb585f30f70f6a
FAIL (2026-05-25 10:05:56 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
e=true , address=212.102.52.216:4399 , piece_id=4F2RKDATBD4ARRQ2W7WJLD6AKHZQEWSQN3BYS2XXS4CTWUCH2PQA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.830098022 (7h28m29.613438131s) [noise=true , address=212.102.52.216:4414 , piece_id=WNAXREPANUDI4NEYQBDUL75SY4Z5MLPGE2ORIOJRQTPVZX2CWAQQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.830837252 (7h28m29.614177351s) [noise=true , address=212.102.52.217:4419 , piece_id=SJW7AM4LK4JSQAQQK2GMSRS3LHSI5EZZBAAIWKYTM7VADEW4MRLA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.831733071 (7h28m29.61507317s) [noise=true , address=212.102.52.215:4385 , piece_id=5TDXQEJVMZLZK3OH53T36LHHVPAMZD2MF6E4YHJMUCFOKU7TMQKQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.832187388 (7h28m29.615527487s) [noise=true , address=212.102.52.213:4343 , piece_id=JJHIT42VPZI6BWRJMC5DBNW32VEIGAN3NWVYDZUN7NN7JMJSXIMA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.832692371 (7h28m29.61603246s) [noise=true , address=212.102.52.213:4324 , piece_id=YMEXBDFUX7FDPUGI4YMKDA3VY2ONIGSFH7QPMTLHC2FNCFVX4EGA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.833491967 (7h28m29.616832066s) [noise=true , address=212.102.52.213:4331 , piece_id=O7TPMVI2SUBJX4PUU7XI4EXVP6B772C6OVIIKGOH4H7BKBV7WFPA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.833985479 (7h28m29.617325578s) [noise=true , address=212.102.52.217:4440 , piece_id=EJGXBWWFIQDR7WVUF3ROPORIV4LFKNYQAHZGPT4GQT4M2ASB2VTA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.836004521 (7h28m29.61934463s) [noise=true , address=212.102.52.217:4422 , piece_id=VVXCZLY77G7RFK25ECZ2UJPAOGWLSELFXB53PIHNIKKCN5TSAB6A, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.838800066 (7h28m29.622140165s) [noise=true , address=212.102.52.217:4430 , piece_id=PN57VWOAPH7U2PPCV5YVQUYPACWLQIDZOP6BRCIPPSCPB23X5ZHQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.840366935 (7h28m29.623707024s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.842006614 (7h28m29.625346713s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.843201753 (7h28m29.626541853s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.845076260 (7h28m29.628416359s) [noise=true , address=212.102.52.214:4364 , piece_id=6ZFNYSVRTKGVJVZIAEYJM5LQUHTVJO4SO5JYGWAISF4TBT4RLTHQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.845505149 (7h28m29.628845258s) [noise=true , address=212.102.52.215:4373 , piece_id=AKUFHKJGCCZ4JB4WGCSJCBLUUI5PAO4F5ZROGHTRUKH7FRRLTOVQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.847289253 (7h28m29.630629352s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:44.897115628 (7h28m30.680455727s) [upload=7184]: Next returned nil segment
May 25 09:32:44.897126990 (7h28m30.680467089s) [upload=7184]: Waiting for error group managing segments...
May 25 09:32:45.181306033 (7h28m30.964646133s) [noise=true , address=212.102.52.216:4414 , piece_id=WNAXREPANUDI4NEYQBDUL75SY4Z5MLPGE2ORIOJRQTPVZX2CWAQQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.182774646 (7h28m30.966114745s) [noise=true , address=212.102.52.216:4408 , piece_id=V2UU3QK5OYPDBL23FUVYCNUTPE6S3BQZL3XWCVAZXZPSECJTRE3Q, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.182945902 (7h28m30.966286001s) [noise=true , address=212.102.52.216:4400 , piece_id=5PJ7CFDLRQE3YPUOATP7O7POFP4NGTRFZKYXVX4KLZTPZULLZ4XA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.185420764 (7h28m30.968760863s) [noise=true , address=212.102.52.216:4399 , piece_id=4F2RKDATBD4ARRQ2W7WJLD6AKHZQEWSQN3BYS2XXS4CTWUCH2PQA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.186002044 (7h28m30.969342143s) [noise=true , address=212.102.52.214:4367 , piece_id=6O5TT7K373VMSSNWNLUVAH7JJPK3Y2MHKQSWYNTULHT26BJJTU2A, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.187072957 (7h28m30.970413056s) [noise=true , address=212.102.52.213:4330 , piece_id=4FP2RPTIETYETJMKUT3XRK7GBKR45TWLNPGJ5ORQYHNTN6LY2DMQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.188495802 (7h28m30.971835901s) [noise=true , address=212.102.52.217:4430 , piece_id=PN57VWOAPH7U2PPCV5YVQUYPACWLQIDZOP6BRCIPPSCPB23X5ZHQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.195292711 (7h28m30.97863281s) [noise=true , address=212.102.52.213:4343 , piece_id=JJHIT42VPZI6BWRJMC5DBNW32VEIGAN3NWVYDZUN7NN7JMJSXIMA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.199852751 (7h28m30.98319285s) [noise=true , address=212.102.52.215:4385 , piece_id=5TDXQEJVMZLZK3OH53T36LHHVPAMZD2MF6E4YHJMUCFOKU7TMQKQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.200159136 (7h28m30.983499225s) [noise=true , address=212.102.52.217:4440 , piece_id=EJGXBWWFIQDR7WVUF3ROPORIV4LFKNYQAHZGPT4GQT4M2ASB2VTA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.203990647 (7h28m30.987330746s) [noise=true , address=212.102.52.213:4331 , piece_id=O7TPMVI2SUBJX4PUU7XI4EXVP6B772C6OVIIKGOH4H7BKBV7WFPA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.204341486 (7h28m30.987681585s) [noise=true , address=212.102.52.215:4373 , piece_id=AKUFHKJGCCZ4JB4WGCSJCBLUUI5PAO4F5ZROGHTRUKH7FRRLTOVQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.207397948 (7h28m30.990738047s) [noise=true , address=212.102.52.217:4422 , piece_id=VVXCZLY77G7RFK25ECZ2UJPAOGWLSELFXB53PIHNIKKCN5TSAB6A, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.209282203 (7h28m30.992622302s) [noise=true , address=212.102.52.213:4324 , piece_id=YMEXBDFUX7FDPUGI4YMKDA3VY2ONIGSFH7QPMTLHC2FNCFVX4EGA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.210214793 (7h28m30.993554882s) [noise=true , address=212.102.52.214:4364 , piece_id=6ZFNYSVRTKGVJVZIAEYJM5LQUHTVJO4SO5JYGWAISF4TBT4RLTHQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.211913804 (7h28m30.995253903s) [noise=true , address=212.102.52.217:4419 , piece_id=SJW7AM4LK4JSQAQQK2GMSRS3LHSI5EZZBAAIWKYTM7VADEW4MRLA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.211924885 (7h28m30.995264984s) [seg_pos={0 0}, upload=7184]: Satisfied cohort requirements after 16 pieces
May 25 09:32:45.212150666 (7h28m30.995490765s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ku2quD8AHh79sZcysV979DNjGBVGAazWZM7P5vPeckaKfoBL5): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212161617 (7h28m30.995501716s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212170323 (7h28m30.995510422s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212177678 (7h28m30.995517777s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212287968 (7h28m30.995628067s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J6FD4Y3i9TV7hgSi3QNR8Bf27t9oenvhecVTgy4cMySGFw6Go): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212292587 (7h28m30.995632676s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212433165 (7h28m30.995773264s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212443646 (7h28m30.995783735s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212461499 (7h28m30.995801598s) [seg_pos={0 0}, upload=7184]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 09:32:45.212537665 (7h28m30.995877764s) [upload=7184]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 09:32:45.212542644 (7h28m30.995882743s) [upload=7184]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 09:32:53.332110792 (7h28m39.115450931s) [upload=7184]: Done uploading segments...
May 25 09:32:53.332134266 (7h28m39.115474395s) [upload=7184]: Upload finished. err: <nil>
Duration: 5.606630475s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
5.013784053s trace:
https://jaeger.dev.storj.io/trace/4eb585f30f70f6a
FAIL (2026-05-25 10:04:41 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
e=true , address=212.102.52.216:4399 , piece_id=4F2RKDATBD4ARRQ2W7WJLD6AKHZQEWSQN3BYS2XXS4CTWUCH2PQA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.830098022 (7h28m29.613438131s) [noise=true , address=212.102.52.216:4414 , piece_id=WNAXREPANUDI4NEYQBDUL75SY4Z5MLPGE2ORIOJRQTPVZX2CWAQQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.830837252 (7h28m29.614177351s) [noise=true , address=212.102.52.217:4419 , piece_id=SJW7AM4LK4JSQAQQK2GMSRS3LHSI5EZZBAAIWKYTM7VADEW4MRLA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.831733071 (7h28m29.61507317s) [noise=true , address=212.102.52.215:4385 , piece_id=5TDXQEJVMZLZK3OH53T36LHHVPAMZD2MF6E4YHJMUCFOKU7TMQKQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.832187388 (7h28m29.615527487s) [noise=true , address=212.102.52.213:4343 , piece_id=JJHIT42VPZI6BWRJMC5DBNW32VEIGAN3NWVYDZUN7NN7JMJSXIMA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.832692371 (7h28m29.61603246s) [noise=true , address=212.102.52.213:4324 , piece_id=YMEXBDFUX7FDPUGI4YMKDA3VY2ONIGSFH7QPMTLHC2FNCFVX4EGA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.833491967 (7h28m29.616832066s) [noise=true , address=212.102.52.213:4331 , piece_id=O7TPMVI2SUBJX4PUU7XI4EXVP6B772C6OVIIKGOH4H7BKBV7WFPA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.833985479 (7h28m29.617325578s) [noise=true , address=212.102.52.217:4440 , piece_id=EJGXBWWFIQDR7WVUF3ROPORIV4LFKNYQAHZGPT4GQT4M2ASB2VTA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.836004521 (7h28m29.61934463s) [noise=true , address=212.102.52.217:4422 , piece_id=VVXCZLY77G7RFK25ECZ2UJPAOGWLSELFXB53PIHNIKKCN5TSAB6A, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.838800066 (7h28m29.622140165s) [noise=true , address=212.102.52.217:4430 , piece_id=PN57VWOAPH7U2PPCV5YVQUYPACWLQIDZOP6BRCIPPSCPB23X5ZHQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.840366935 (7h28m29.623707024s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.842006614 (7h28m29.625346713s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.843201753 (7h28m29.626541853s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.845076260 (7h28m29.628416359s) [noise=true , address=212.102.52.214:4364 , piece_id=6ZFNYSVRTKGVJVZIAEYJM5LQUHTVJO4SO5JYGWAISF4TBT4RLTHQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.845505149 (7h28m29.628845258s) [noise=true , address=212.102.52.215:4373 , piece_id=AKUFHKJGCCZ4JB4WGCSJCBLUUI5PAO4F5ZROGHTRUKH7FRRLTOVQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.847289253 (7h28m29.630629352s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:44.897115628 (7h28m30.680455727s) [upload=7184]: Next returned nil segment
May 25 09:32:44.897126990 (7h28m30.680467089s) [upload=7184]: Waiting for error group managing segments...
May 25 09:32:45.181306033 (7h28m30.964646133s) [noise=true , address=212.102.52.216:4414 , piece_id=WNAXREPANUDI4NEYQBDUL75SY4Z5MLPGE2ORIOJRQTPVZX2CWAQQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.182774646 (7h28m30.966114745s) [noise=true , address=212.102.52.216:4408 , piece_id=V2UU3QK5OYPDBL23FUVYCNUTPE6S3BQZL3XWCVAZXZPSECJTRE3Q, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.182945902 (7h28m30.966286001s) [noise=true , address=212.102.52.216:4400 , piece_id=5PJ7CFDLRQE3YPUOATP7O7POFP4NGTRFZKYXVX4KLZTPZULLZ4XA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.185420764 (7h28m30.968760863s) [noise=true , address=212.102.52.216:4399 , piece_id=4F2RKDATBD4ARRQ2W7WJLD6AKHZQEWSQN3BYS2XXS4CTWUCH2PQA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.186002044 (7h28m30.969342143s) [noise=true , address=212.102.52.214:4367 , piece_id=6O5TT7K373VMSSNWNLUVAH7JJPK3Y2MHKQSWYNTULHT26BJJTU2A, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.187072957 (7h28m30.970413056s) [noise=true , address=212.102.52.213:4330 , piece_id=4FP2RPTIETYETJMKUT3XRK7GBKR45TWLNPGJ5ORQYHNTN6LY2DMQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.188495802 (7h28m30.971835901s) [noise=true , address=212.102.52.217:4430 , piece_id=PN57VWOAPH7U2PPCV5YVQUYPACWLQIDZOP6BRCIPPSCPB23X5ZHQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.195292711 (7h28m30.97863281s) [noise=true , address=212.102.52.213:4343 , piece_id=JJHIT42VPZI6BWRJMC5DBNW32VEIGAN3NWVYDZUN7NN7JMJSXIMA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.199852751 (7h28m30.98319285s) [noise=true , address=212.102.52.215:4385 , piece_id=5TDXQEJVMZLZK3OH53T36LHHVPAMZD2MF6E4YHJMUCFOKU7TMQKQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.200159136 (7h28m30.983499225s) [noise=true , address=212.102.52.217:4440 , piece_id=EJGXBWWFIQDR7WVUF3ROPORIV4LFKNYQAHZGPT4GQT4M2ASB2VTA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.203990647 (7h28m30.987330746s) [noise=true , address=212.102.52.213:4331 , piece_id=O7TPMVI2SUBJX4PUU7XI4EXVP6B772C6OVIIKGOH4H7BKBV7WFPA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.204341486 (7h28m30.987681585s) [noise=true , address=212.102.52.215:4373 , piece_id=AKUFHKJGCCZ4JB4WGCSJCBLUUI5PAO4F5ZROGHTRUKH7FRRLTOVQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.207397948 (7h28m30.990738047s) [noise=true , address=212.102.52.217:4422 , piece_id=VVXCZLY77G7RFK25ECZ2UJPAOGWLSELFXB53PIHNIKKCN5TSAB6A, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.209282203 (7h28m30.992622302s) [noise=true , address=212.102.52.213:4324 , piece_id=YMEXBDFUX7FDPUGI4YMKDA3VY2ONIGSFH7QPMTLHC2FNCFVX4EGA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.210214793 (7h28m30.993554882s) [noise=true , address=212.102.52.214:4364 , piece_id=6ZFNYSVRTKGVJVZIAEYJM5LQUHTVJO4SO5JYGWAISF4TBT4RLTHQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.211913804 (7h28m30.995253903s) [noise=true , address=212.102.52.217:4419 , piece_id=SJW7AM4LK4JSQAQQK2GMSRS3LHSI5EZZBAAIWKYTM7VADEW4MRLA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.211924885 (7h28m30.995264984s) [seg_pos={0 0}, upload=7184]: Satisfied cohort requirements after 16 pieces
May 25 09:32:45.212150666 (7h28m30.995490765s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ku2quD8AHh79sZcysV979DNjGBVGAazWZM7P5vPeckaKfoBL5): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212161617 (7h28m30.995501716s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212170323 (7h28m30.995510422s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212177678 (7h28m30.995517777s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212287968 (7h28m30.995628067s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J6FD4Y3i9TV7hgSi3QNR8Bf27t9oenvhecVTgy4cMySGFw6Go): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212292587 (7h28m30.995632676s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212433165 (7h28m30.995773264s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212443646 (7h28m30.995783735s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212461499 (7h28m30.995801598s) [seg_pos={0 0}, upload=7184]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 09:32:45.212537665 (7h28m30.995877764s) [upload=7184]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 09:32:45.212542644 (7h28m30.995882743s) [upload=7184]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 09:32:53.332110792 (7h28m39.115450931s) [upload=7184]: Done uploading segments...
May 25 09:32:53.332134266 (7h28m39.115474395s) [upload=7184]: Upload finished. err: <nil>
Duration: 5.013812185s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
4.352833166s trace:
https://jaeger.dev.storj.io/trace/4eb585f30f70f6a
FAIL (2026-05-25 10:03:53 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
e=true , address=212.102.52.216:4399 , piece_id=4F2RKDATBD4ARRQ2W7WJLD6AKHZQEWSQN3BYS2XXS4CTWUCH2PQA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.830098022 (7h28m29.613438131s) [noise=true , address=212.102.52.216:4414 , piece_id=WNAXREPANUDI4NEYQBDUL75SY4Z5MLPGE2ORIOJRQTPVZX2CWAQQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.830837252 (7h28m29.614177351s) [noise=true , address=212.102.52.217:4419 , piece_id=SJW7AM4LK4JSQAQQK2GMSRS3LHSI5EZZBAAIWKYTM7VADEW4MRLA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.831733071 (7h28m29.61507317s) [noise=true , address=212.102.52.215:4385 , piece_id=5TDXQEJVMZLZK3OH53T36LHHVPAMZD2MF6E4YHJMUCFOKU7TMQKQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.832187388 (7h28m29.615527487s) [noise=true , address=212.102.52.213:4343 , piece_id=JJHIT42VPZI6BWRJMC5DBNW32VEIGAN3NWVYDZUN7NN7JMJSXIMA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.832692371 (7h28m29.61603246s) [noise=true , address=212.102.52.213:4324 , piece_id=YMEXBDFUX7FDPUGI4YMKDA3VY2ONIGSFH7QPMTLHC2FNCFVX4EGA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.833491967 (7h28m29.616832066s) [noise=true , address=212.102.52.213:4331 , piece_id=O7TPMVI2SUBJX4PUU7XI4EXVP6B772C6OVIIKGOH4H7BKBV7WFPA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.833985479 (7h28m29.617325578s) [noise=true , address=212.102.52.217:4440 , piece_id=EJGXBWWFIQDR7WVUF3ROPORIV4LFKNYQAHZGPT4GQT4M2ASB2VTA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.836004521 (7h28m29.61934463s) [noise=true , address=212.102.52.217:4422 , piece_id=VVXCZLY77G7RFK25ECZ2UJPAOGWLSELFXB53PIHNIKKCN5TSAB6A, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.838800066 (7h28m29.622140165s) [noise=true , address=212.102.52.217:4430 , piece_id=PN57VWOAPH7U2PPCV5YVQUYPACWLQIDZOP6BRCIPPSCPB23X5ZHQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.840366935 (7h28m29.623707024s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.842006614 (7h28m29.625346713s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.843201753 (7h28m29.626541853s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.845076260 (7h28m29.628416359s) [noise=true , address=212.102.52.214:4364 , piece_id=6ZFNYSVRTKGVJVZIAEYJM5LQUHTVJO4SO5JYGWAISF4TBT4RLTHQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.845505149 (7h28m29.628845258s) [noise=true , address=212.102.52.215:4373 , piece_id=AKUFHKJGCCZ4JB4WGCSJCBLUUI5PAO4F5ZROGHTRUKH7FRRLTOVQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.847289253 (7h28m29.630629352s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:44.897115628 (7h28m30.680455727s) [upload=7184]: Next returned nil segment
May 25 09:32:44.897126990 (7h28m30.680467089s) [upload=7184]: Waiting for error group managing segments...
May 25 09:32:45.181306033 (7h28m30.964646133s) [noise=true , address=212.102.52.216:4414 , piece_id=WNAXREPANUDI4NEYQBDUL75SY4Z5MLPGE2ORIOJRQTPVZX2CWAQQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.182774646 (7h28m30.966114745s) [noise=true , address=212.102.52.216:4408 , piece_id=V2UU3QK5OYPDBL23FUVYCNUTPE6S3BQZL3XWCVAZXZPSECJTRE3Q, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.182945902 (7h28m30.966286001s) [noise=true , address=212.102.52.216:4400 , piece_id=5PJ7CFDLRQE3YPUOATP7O7POFP4NGTRFZKYXVX4KLZTPZULLZ4XA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.185420764 (7h28m30.968760863s) [noise=true , address=212.102.52.216:4399 , piece_id=4F2RKDATBD4ARRQ2W7WJLD6AKHZQEWSQN3BYS2XXS4CTWUCH2PQA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.186002044 (7h28m30.969342143s) [noise=true , address=212.102.52.214:4367 , piece_id=6O5TT7K373VMSSNWNLUVAH7JJPK3Y2MHKQSWYNTULHT26BJJTU2A, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.187072957 (7h28m30.970413056s) [noise=true , address=212.102.52.213:4330 , piece_id=4FP2RPTIETYETJMKUT3XRK7GBKR45TWLNPGJ5ORQYHNTN6LY2DMQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.188495802 (7h28m30.971835901s) [noise=true , address=212.102.52.217:4430 , piece_id=PN57VWOAPH7U2PPCV5YVQUYPACWLQIDZOP6BRCIPPSCPB23X5ZHQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.195292711 (7h28m30.97863281s) [noise=true , address=212.102.52.213:4343 , piece_id=JJHIT42VPZI6BWRJMC5DBNW32VEIGAN3NWVYDZUN7NN7JMJSXIMA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.199852751 (7h28m30.98319285s) [noise=true , address=212.102.52.215:4385 , piece_id=5TDXQEJVMZLZK3OH53T36LHHVPAMZD2MF6E4YHJMUCFOKU7TMQKQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.200159136 (7h28m30.983499225s) [noise=true , address=212.102.52.217:4440 , piece_id=EJGXBWWFIQDR7WVUF3ROPORIV4LFKNYQAHZGPT4GQT4M2ASB2VTA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.203990647 (7h28m30.987330746s) [noise=true , address=212.102.52.213:4331 , piece_id=O7TPMVI2SUBJX4PUU7XI4EXVP6B772C6OVIIKGOH4H7BKBV7WFPA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.204341486 (7h28m30.987681585s) [noise=true , address=212.102.52.215:4373 , piece_id=AKUFHKJGCCZ4JB4WGCSJCBLUUI5PAO4F5ZROGHTRUKH7FRRLTOVQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.207397948 (7h28m30.990738047s) [noise=true , address=212.102.52.217:4422 , piece_id=VVXCZLY77G7RFK25ECZ2UJPAOGWLSELFXB53PIHNIKKCN5TSAB6A, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.209282203 (7h28m30.992622302s) [noise=true , address=212.102.52.213:4324 , piece_id=YMEXBDFUX7FDPUGI4YMKDA3VY2ONIGSFH7QPMTLHC2FNCFVX4EGA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.210214793 (7h28m30.993554882s) [noise=true , address=212.102.52.214:4364 , piece_id=6ZFNYSVRTKGVJVZIAEYJM5LQUHTVJO4SO5JYGWAISF4TBT4RLTHQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.211913804 (7h28m30.995253903s) [noise=true , address=212.102.52.217:4419 , piece_id=SJW7AM4LK4JSQAQQK2GMSRS3LHSI5EZZBAAIWKYTM7VADEW4MRLA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.211924885 (7h28m30.995264984s) [seg_pos={0 0}, upload=7184]: Satisfied cohort requirements after 16 pieces
May 25 09:32:45.212150666 (7h28m30.995490765s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ku2quD8AHh79sZcysV979DNjGBVGAazWZM7P5vPeckaKfoBL5): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212161617 (7h28m30.995501716s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212170323 (7h28m30.995510422s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212177678 (7h28m30.995517777s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212287968 (7h28m30.995628067s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J6FD4Y3i9TV7hgSi3QNR8Bf27t9oenvhecVTgy4cMySGFw6Go): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212292587 (7h28m30.995632676s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212433165 (7h28m30.995773264s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212443646 (7h28m30.995783735s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212461499 (7h28m30.995801598s) [seg_pos={0 0}, upload=7184]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 09:32:45.212537665 (7h28m30.995877764s) [upload=7184]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 09:32:45.212542644 (7h28m30.995882743s) [upload=7184]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 09:32:53.332110792 (7h28m39.115450931s) [upload=7184]: Done uploading segments...
May 25 09:32:53.332134266 (7h28m39.115474395s) [upload=7184]: Upload finished. err: <nil>
Duration: 4.352853776s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
4.310631602s trace:
https://jaeger.dev.storj.io/trace/4eb585f30f70f6a
FAIL (2026-05-25 10:02:45 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
e=true , address=212.102.52.216:4399 , piece_id=4F2RKDATBD4ARRQ2W7WJLD6AKHZQEWSQN3BYS2XXS4CTWUCH2PQA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.830098022 (7h28m29.613438131s) [noise=true , address=212.102.52.216:4414 , piece_id=WNAXREPANUDI4NEYQBDUL75SY4Z5MLPGE2ORIOJRQTPVZX2CWAQQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.830837252 (7h28m29.614177351s) [noise=true , address=212.102.52.217:4419 , piece_id=SJW7AM4LK4JSQAQQK2GMSRS3LHSI5EZZBAAIWKYTM7VADEW4MRLA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.831733071 (7h28m29.61507317s) [noise=true , address=212.102.52.215:4385 , piece_id=5TDXQEJVMZLZK3OH53T36LHHVPAMZD2MF6E4YHJMUCFOKU7TMQKQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.832187388 (7h28m29.615527487s) [noise=true , address=212.102.52.213:4343 , piece_id=JJHIT42VPZI6BWRJMC5DBNW32VEIGAN3NWVYDZUN7NN7JMJSXIMA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.832692371 (7h28m29.61603246s) [noise=true , address=212.102.52.213:4324 , piece_id=YMEXBDFUX7FDPUGI4YMKDA3VY2ONIGSFH7QPMTLHC2FNCFVX4EGA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.833491967 (7h28m29.616832066s) [noise=true , address=212.102.52.213:4331 , piece_id=O7TPMVI2SUBJX4PUU7XI4EXVP6B772C6OVIIKGOH4H7BKBV7WFPA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.833985479 (7h28m29.617325578s) [noise=true , address=212.102.52.217:4440 , piece_id=EJGXBWWFIQDR7WVUF3ROPORIV4LFKNYQAHZGPT4GQT4M2ASB2VTA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.836004521 (7h28m29.61934463s) [noise=true , address=212.102.52.217:4422 , piece_id=VVXCZLY77G7RFK25ECZ2UJPAOGWLSELFXB53PIHNIKKCN5TSAB6A, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.838800066 (7h28m29.622140165s) [noise=true , address=212.102.52.217:4430 , piece_id=PN57VWOAPH7U2PPCV5YVQUYPACWLQIDZOP6BRCIPPSCPB23X5ZHQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.840366935 (7h28m29.623707024s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.842006614 (7h28m29.625346713s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.843201753 (7h28m29.626541853s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.845076260 (7h28m29.628416359s) [noise=true , address=212.102.52.214:4364 , piece_id=6ZFNYSVRTKGVJVZIAEYJM5LQUHTVJO4SO5JYGWAISF4TBT4RLTHQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.845505149 (7h28m29.628845258s) [noise=true , address=212.102.52.215:4373 , piece_id=AKUFHKJGCCZ4JB4WGCSJCBLUUI5PAO4F5ZROGHTRUKH7FRRLTOVQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:43.847289253 (7h28m29.630629352s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Uploading piece...
May 25 09:32:44.897115628 (7h28m30.680455727s) [upload=7184]: Next returned nil segment
May 25 09:32:44.897126990 (7h28m30.680467089s) [upload=7184]: Waiting for error group managing segments...
May 25 09:32:45.181306033 (7h28m30.964646133s) [noise=true , address=212.102.52.216:4414 , piece_id=WNAXREPANUDI4NEYQBDUL75SY4Z5MLPGE2ORIOJRQTPVZX2CWAQQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.182774646 (7h28m30.966114745s) [noise=true , address=212.102.52.216:4408 , piece_id=V2UU3QK5OYPDBL23FUVYCNUTPE6S3BQZL3XWCVAZXZPSECJTRE3Q, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.182945902 (7h28m30.966286001s) [noise=true , address=212.102.52.216:4400 , piece_id=5PJ7CFDLRQE3YPUOATP7O7POFP4NGTRFZKYXVX4KLZTPZULLZ4XA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.185420764 (7h28m30.968760863s) [noise=true , address=212.102.52.216:4399 , piece_id=4F2RKDATBD4ARRQ2W7WJLD6AKHZQEWSQN3BYS2XXS4CTWUCH2PQA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.186002044 (7h28m30.969342143s) [noise=true , address=212.102.52.214:4367 , piece_id=6O5TT7K373VMSSNWNLUVAH7JJPK3Y2MHKQSWYNTULHT26BJJTU2A, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.187072957 (7h28m30.970413056s) [noise=true , address=212.102.52.213:4330 , piece_id=4FP2RPTIETYETJMKUT3XRK7GBKR45TWLNPGJ5ORQYHNTN6LY2DMQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.188495802 (7h28m30.971835901s) [noise=true , address=212.102.52.217:4430 , piece_id=PN57VWOAPH7U2PPCV5YVQUYPACWLQIDZOP6BRCIPPSCPB23X5ZHQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.195292711 (7h28m30.97863281s) [noise=true , address=212.102.52.213:4343 , piece_id=JJHIT42VPZI6BWRJMC5DBNW32VEIGAN3NWVYDZUN7NN7JMJSXIMA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.199852751 (7h28m30.98319285s) [noise=true , address=212.102.52.215:4385 , piece_id=5TDXQEJVMZLZK3OH53T36LHHVPAMZD2MF6E4YHJMUCFOKU7TMQKQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.200159136 (7h28m30.983499225s) [noise=true , address=212.102.52.217:4440 , piece_id=EJGXBWWFIQDR7WVUF3ROPORIV4LFKNYQAHZGPT4GQT4M2ASB2VTA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.203990647 (7h28m30.987330746s) [noise=true , address=212.102.52.213:4331 , piece_id=O7TPMVI2SUBJX4PUU7XI4EXVP6B772C6OVIIKGOH4H7BKBV7WFPA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.204341486 (7h28m30.987681585s) [noise=true , address=212.102.52.215:4373 , piece_id=AKUFHKJGCCZ4JB4WGCSJCBLUUI5PAO4F5ZROGHTRUKH7FRRLTOVQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.207397948 (7h28m30.990738047s) [noise=true , address=212.102.52.217:4422 , piece_id=VVXCZLY77G7RFK25ECZ2UJPAOGWLSELFXB53PIHNIKKCN5TSAB6A, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.209282203 (7h28m30.992622302s) [noise=true , address=212.102.52.213:4324 , piece_id=YMEXBDFUX7FDPUGI4YMKDA3VY2ONIGSFH7QPMTLHC2FNCFVX4EGA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.210214793 (7h28m30.993554882s) [noise=true , address=212.102.52.214:4364 , piece_id=6ZFNYSVRTKGVJVZIAEYJM5LQUHTVJO4SO5JYGWAISF4TBT4RLTHQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.211913804 (7h28m30.995253903s) [noise=true , address=212.102.52.217:4419 , piece_id=SJW7AM4LK4JSQAQQK2GMSRS3LHSI5EZZBAAIWKYTM7VADEW4MRLA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: <nil>
May 25 09:32:45.211924885 (7h28m30.995264984s) [seg_pos={0 0}, upload=7184]: Satisfied cohort requirements after 16 pieces
May 25 09:32:45.212150666 (7h28m30.995490765s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ku2quD8AHh79sZcysV979DNjGBVGAazWZM7P5vPeckaKfoBL5): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212161617 (7h28m30.995501716s) [noise=true , address=212.102.52.214:4345 , piece_id=ESRBASLBWMHSBDHNH3JVKJD7QLBSFIQVEGGVHYXYEIDQOSBJ4BBA, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212170323 (7h28m30.995510422s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212177678 (7h28m30.995517777s) [noise=true , address=212.102.52.215:4382 , piece_id=SFMPBZMEX6SH2GWMZETW5XL5PHV256F6L64MR2RC66C24IFADDLQ, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212287968 (7h28m30.995628067s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J6FD4Y3i9TV7hgSi3QNR8Bf27t9oenvhecVTgy4cMySGFw6Go): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212292587 (7h28m30.995632676s) [noise=true , address=212.102.52.215:4377 , piece_id=32TS6FJTMIPFAJFNPRC7TYQH5WA7PTIM7RMARSFMFFUMCQGGA3UA, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212433165 (7h28m30.995773264s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 09:32:45.212443646 (7h28m30.995783735s) [noise=true , address=212.102.52.214:4368 , piece_id=5EPA362ZLVJKC7BJ6GBAHS65KBPZRZLBJU4SCGPVOBCIMKUUXH3Q, seg_pos={0 0}, upload=7184]: Long tail cancellation due to optimal threshold
May 25 09:32:45.212461499 (7h28m30.995801598s) [seg_pos={0 0}, upload=7184]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 09:32:45.212537665 (7h28m30.995877764s) [upload=7184]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 09:32:45.212542644 (7h28m30.995882743s) [upload=7184]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 09:32:53.332110792 (7h28m39.115450931s) [upload=7184]: Done uploading segments...
May 25 09:32:53.332134266 (7h28m39.115474395s) [upload=7184]: Upload finished. err: <nil>
Duration: 4.310653976s
core
uplink
uk-select-1
uk1-placement-17
|