Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.45823183s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:34:04 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.458258692s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.277631832s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:33:40 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.277650509s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.359245205s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:32:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.359271556s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.709635729s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:31:35 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.709693321s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.506549977s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:30:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.506571659s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.336923438s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:29:33 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.336943577s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.39496293s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:28:27 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.394987698s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.386475747s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:27:38 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.386504071s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.217061385s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:26:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.217091965s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.62602105s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:26:19 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.626045616s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.534546177s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:25:48 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.53457382s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.18413339s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:25:15 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.184160903s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.392390344s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:24:03 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.392413467s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.405051373s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:23:00 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.405075509s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.514976923s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:22:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.515002832s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.5317971s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:21:10 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.531823261s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.873700564s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:20:34 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.873731805s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.70434902s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:20:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.704378367s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.567632519s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:19:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.567658378s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.461449238s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:18:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.461467313s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.392617459s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:18:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.392660463s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.454739728s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:17:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.454768394s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 7.30853242s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:17:10 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 7.308557929s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.417901474s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:15:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.4179298s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.468452152s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:15:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.468503571s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.470219606s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:14:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.470245496s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.524751305s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:13:44 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.524774441s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.489665385s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:13:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.489694411s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.198236824s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:12:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.198294435s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.391633835s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:11:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.391664023s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.788886156s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:11:27 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.7889145s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.425331397s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:10:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.42535868s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.410999597s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:09:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.411028513s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.510640471s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:09:16 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.510663246s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.581460522s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:08:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.581488647s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.406195007s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:08:08 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.406215125s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.522651831s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:07:06 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.522678914s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.705168676s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:06:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.705196771s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.355641636s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:06:25 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.35566938s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.265192195s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:05:31 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.265218776s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.180750276s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:04:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.180776567s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.334416369s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:03:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.334445475s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.463056535s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:02:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.463086512s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.348225889s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:02:02 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.348243432s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.395303866s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:01:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.395334375s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.633353152s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:01:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.633375143s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.546203101s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:00:12 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.546228359s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.385375948s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 20:59:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.385414343s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.300313476s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 20:58:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.300338523s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.171349842s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 20:58:17 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.171380531s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)