Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.873700564s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:20:34 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.873731805s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.70434902s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:20:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.704378367s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.567632519s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:19:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.567658378s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.461449238s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:18:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.461467313s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.392617459s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:18:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.392660463s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.454739728s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:17:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.454768394s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 7.30853242s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:17:10 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 7.308557929s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.417901474s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:15:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.4179298s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.468452152s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:15:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.468503571s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.470219606s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:14:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.470245496s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.524751305s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:13:44 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.524774441s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.489665385s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:13:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.489694411s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.198236824s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:12:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.198294435s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.391633835s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:11:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.391664023s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.788886156s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:11:27 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.7889145s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.425331397s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:10:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.42535868s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.410999597s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:09:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.411028513s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.510640471s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:09:16 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.510663246s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.581460522s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:08:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.581488647s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.406195007s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:08:08 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.406215125s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.522651831s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:07:06 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.522678914s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.705168676s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:06:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.705196771s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.355641636s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:06:25 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.35566938s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.265192195s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:05:31 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.265218776s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.180750276s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:04:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.180776567s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.334416369s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:03:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.334445475s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.463056535s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:02:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.463086512s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.348225889s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:02:02 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.348243432s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.395303866s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:01:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.395334375s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.633353152s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:01:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.633375143s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.546203101s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 21:00:12 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.546228359s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.385375948s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 20:59:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.385414343s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.300313476s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 20:58:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.300338523s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.171349842s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 20:58:17 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 6.171380531s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.296940247s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 20:57:49 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.29696756s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.34888903s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 20:57:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.348913967s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.572679167s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 20:56:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 5.57270667s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 4.178186403s trace: https://jaeger.dev.storj.io/trace/7587bb35b90f6d28 FAIL (2026-04-05 20:55:55 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.725894561 (729h50m34.638232192s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726191995 (729h50m34.638529626s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726468208 (729h50m34.638805839s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726728941 (729h50m34.639066572s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.726782515 (729h50m34.639120146s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727039180 (729h50m34.639376811s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727340321 (729h50m34.639677952s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.727686359 (729h50m34.64002399s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728080840 (729h50m34.640418471s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728387341 (729h50m34.640724972s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.728817081 (729h50m34.641154712s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729344909 (729h50m34.64168254s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.729668374 (729h50m34.642006005s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730198978 (729h50m34.642536609s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:51.730547460 (729h50m34.642885091s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Uploading piece... Apr 5 20:55:53.082105812 (729h50m35.994443444s) [upload=700380]: Next returned nil segment Apr 5 20:55:53.082120361 (729h50m35.994457992s) [upload=700380]: Waiting for error group managing segments... Apr 5 20:55:53.450200301 (729h50m36.362537932s) [noise=true , address=167.17.72.70:4762 , piece_id=MDVUNLWQON6RKHXAFK5MZD7GHXX64KGEBWLLQDSXRSLCXMFJCZYA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.490511347 (729h50m36.402848988s) [noise=true , address=193.149.164.71:4292 , piece_id=MF53WZFDJNHOFS5DOTYG2VHSR7HD2UFN3QGJ7EMEXMHVZYVQWXOQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.506882072 (729h50m36.419219703s) [noise=true , address=167.17.72.71:4795 , piece_id=3YK7I2UBS4FK3PWW2UFXAYXAQVLHV4SP3YMNDQGN2VBEBNZLO46Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.508907601 (729h50m36.421245242s) [noise=true , address=193.149.164.70:4107 , piece_id=PMKMMFTTUGJTPN2QISILSMD3G4QGRDCK4YZYC5YAWQQXFBAJQ7AQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.525357668 (729h50m36.437695299s) [noise=true , address=193.149.164.68:4043 , piece_id=YO6VHATBTNLHOYF2DIICMVYCXANJVPCRWNJL2UIE7VZXXKULEJTA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.664794879 (729h50m36.57713251s) [noise=true , address=45.91.116.11:4143 , piece_id=2KGQ3WPZES3FITLU7PJJJSNPCDTDV3732IJB3J7YEXVYEAZ5CPMQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.665301246 (729h50m36.577638877s) [noise=true , address=77.90.31.19:4862 , piece_id=C4T7OAXQ6SJI4FH5NFMBQNK2IQEB2WUWFJFUA2J3DNQDUGWLJZXQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.676395888 (729h50m36.588733519s) [noise=true , address=145.79.227.5:4244 , piece_id=N6ZSZUQEPRRAZXQPF67AJZQQBQH76P24YDDTLVPBZYIRA4PVBR2Q, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.680001156 (729h50m36.592338787s) [noise=true , address=77.90.31.12:4603 , piece_id=LBJLT76AMDNXVK5BQYQ2V2ZHOUQ52HRPIC3IHX7KIGPL4T5E2AWA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.699631991 (729h50m36.611969622s) [noise=true , address=77.90.31.13:4655 , piece_id=633UNA4NQAJ2QOSKSEA5QAMVLAPJWYIQWNKZYTSR77RWPORUWLDA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.740105361 (729h50m36.652443022s) [noise=true , address=145.79.227.32:4857 , piece_id=VSG2XLMYW2AYSFHOVVNUO2HVKOOEPIGT6YZFU6KHLVM622H42WEQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.744711751 (729h50m36.657049382s) [noise=true , address=193.149.164.67:4023 , piece_id=ZNAU3DQRI5RNEG66A7KM2HQWWZH3WWF74T3DUCEPXJEM2U7I5HEA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.750295467 (729h50m36.662633098s) [noise=true , address=145.79.227.11:4751 , piece_id=SYS7EPSWNVAM76LJLWXV6QCPBHADLRDF324IUY7IZKWKHOPNYMAQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755085932 (729h50m36.667423573s) [noise=true , address=145.79.227.28:4760 , piece_id=4QAL2EORUHFJRFSB5F5PIA27QKNMKM3ICF3BVYLLWM2RJPXWINWQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: <nil> Apr 5 20:55:53.755099238 (729h50m36.667436869s) [seg_pos={0 0}, upload=700380]: Satisfied cohort requirements after 14 pieces Apr 5 20:55:53.755287020 (729h50m36.667624651s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LVXge8vYMEqzyAF4NWhqQBtw4KXUPXcTPTUMwViQuZkXetkJV): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755302259 (729h50m36.667639911s) [noise=true , address=45.91.116.5:4313 , piece_id=HOTLNBUSQG2SRNRCMTHGVJG3SP2FOBUPC5YYANVLJXJOA4GKHBLQ, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755332779 (729h50m36.66767041s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SsKS8mvgkHszRNtGrt5bktXsJpbdggkeY4tuiAUni6hZ3FnAw): protocol: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.19:4093: use of closed network connection Apr 5 20:55:53.755341756 (729h50m36.667679387s) [noise=true , address=145.79.227.19:4093 , piece_id=ZGE6WAZVHEPMIV7SMBR53ZMNN2RZAM2OWOHUW5L75BD6XP3AE2YA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755463421 (729h50m36.667801052s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755469592 (729h50m36.667807223s) [noise=true , address=145.79.227.30:4768 , piece_id=UWWA4KI5Q4XTI6LTPPO52IB6LLGWK7A7RIKM3U5NPGC7KOTUTZBA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755567621 (729h50m36.667905252s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Apr 5 20:55:53.755572401 (729h50m36.667910032s) [noise=true , address=145.79.227.13:4836 , piece_id=H6YSZJ6EC5ASESMCENTXVCXRQW72YYSDOL2RSFKNANH57ZCNX6UA, seg_pos={0 0}, upload=700380]: Long tail cancellation due to optimal threshold Apr 5 20:55:53.755585566 (729h50m36.667923197s) [seg_pos={0 0}, upload=700380]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 5 20:55:53.755646444 (729h50m36.667984075s) [upload=700380]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 5 20:55:53.755650922 (729h50m36.667988553s) [upload=700380]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 5 20:55:54.049150509 (729h50m36.96148814s) [upload=700380]: Done uploading segments... Apr 5 20:55:54.049169485 (729h50m36.961507116s) [upload=700380]: Upload finished. err: <nil> Duration: 4.178220258s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 4.530196476s trace: https://jaeger.dev.storj.io/trace/71e8b0fffa13491b FAIL (2026-04-05 20:06:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== 3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.445047209 (728h18m29.35738484s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.446660112 (728h18m29.358997743s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:47.230997446 (728h18m30.143335087s) [upload=698890]: Next returned nil segment Apr 5 19:23:47.231015220 (728h18m30.143352852s) [upload=698890]: Waiting for error group managing segments... Apr 5 19:23:47.620452706 (728h18m30.532790347s) [noise=true , address=185.167.204.205:4003 , piece_id=POAEFTZYX3ULC346GTXGT35G2C3CNGGKDBDH24MMLG3L5K6XEUQQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620924707 (728h18m30.533262338s) [noise=true , address=185.167.204.205:4006 , piece_id=OH4TLM4XGF4RFSIMJUCOZTRZAJ2VXSD5IZ272D5O4ZTRJV2EKXFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620995212 (728h18m30.533332843s) [noise=true , address=185.167.204.205:4036 , piece_id=VNGVFT6CPVAFI22JM7YP632ZHFHF63TYOS5OHMPN6SIJYI47Z6ZA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.623436224 (728h18m30.535773855s) [noise=true , address=185.167.204.206:4052 , piece_id=MURH7U3OF7FDWXLNFX2D62OLQXZBRHF5IH2B7O2NIDHYXYVMWPZQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.627891562 (728h18m30.540229193s) [noise=true , address=185.167.204.206:4064 , piece_id=R7LLF3HEHD6HVO4YOBXMNVUSXSOHZL4BDX2RSIL2SKGS3YLPJWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.628832688 (728h18m30.541170319s) [noise=true , address=185.167.204.206:4057 , piece_id=FVDAYDB2M3BS75XT6IWQRSP2QUEVFHSUF3EYAYHHQP37Z2OHS5XA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.629152575 (728h18m30.541490206s) [noise=true , address=185.167.204.206:4069 , piece_id=MWVVC2BOZMC4TWXPVELWHD4RZTBSMYR53CJSIPGQ25B7VDBTMQIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641386105 (728h18m30.553723736s) [noise=true , address=185.167.204.206:4043 , piece_id=VZKY2PYLTFYRGF2JGYWNSHRPEAX6GYTKW7ASDQL7ZALAXXXKNYFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641788261 (728h18m30.554125902s) [noise=true , address=185.167.204.205:4024 , piece_id=2N453JCGF4QS3LKGEZ7BDQZ45NRDPUTBZWECWUKOYBPBRLHJUD3Q, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.643005260 (728h18m30.555342891s) [noise=true , address=185.167.204.205:4011 , piece_id=BG66DKR26PWXMAFTAONWEUFPU6SMOOKMIZWNZSGOIRX2IMUJR4FQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.645249922 (728h18m30.557587553s) [noise=true , address=185.167.204.206:4065 , piece_id=EFD2RBNEIUL4XJ3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651359453 (728h18m30.563697084s) [noise=true , address=185.167.204.206:4056 , piece_id=J5L4SVTQZY7WSPSQ2GXOV7MDN4DIYE454N6FT6K7ZGUD7HR2BFHA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651412215 (728h18m30.563749846s) [noise=true , address=185.167.204.205:4034 , piece_id=VNTYO6Q4IMH2T2XEHJWH64F3YUL45BBCNQLZ46WUZAX62V2SMFTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.654801867 (728h18m30.567139498s) [noise=true , address=185.167.204.206:4071 , piece_id=5NSML2UJOWKUN3FUGXXD37NU3MARIKY6TDCLCDXC6S6WVFV7C5RA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.655101795 (728h18m30.567439426s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.658113638 (728h18m30.570451269s) [noise=true , address=185.167.204.206:4055 , piece_id=VIYGDMLV7V3V76NZ3V5IWBICO6CT7YNTP4BN5QHI542HNFVJ3PTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.662924132 (728h18m30.575261763s) [noise=true , address=185.167.204.206:4062 , piece_id=GRRUPYUEBCY2S6JPGJOEXNJ3OZZDZBYVX27AEMW5YV6IKGEZ6ZQA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.666865058 (728h18m30.579202689s) [noise=true , address=185.167.204.205:4033 , piece_id=EO2VKCTXXE7MA4UOPWABIV23LJDKGMKIUNV3KGFS47OIVXB5WVKQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.670168473 (728h18m30.582506104s) [noise=true , address=185.167.204.205:4009 , piece_id=YQ6P3DOBAPZKVCVTTBI2IJH2UMWWLMLWXMIJNRNSADDSHYF2KGAQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:51.907352601 (728h18m34.819690232s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:24:45.775054746 (728h19m28.687392447s) [noise=true , address=89.149.201.147:4011 , piece_id=FZK5PMW5EYIQLVOLCUOIWR2S42I7BLHX6FRV2MFOTPUKYF36GH2A, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection Apr 5 19:24:45.775168405 (728h19m28.687506076s) [noise=true , address=81.171.14.70:4048 , piece_id=MUDLJPWEYZYTMXLJJZ4FFOTGGVMXZVJSHXE5FZPNHE2SIW6LKPLA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 19:24:45.775267757 (728h19m28.687605408s) [noise=true , address=81.171.14.70:4045 , piece_id=3S6DJCM2FVYHVTFA64VBAWELYHHNKC6RZXBLCUUJS7XY3GACRPIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection Apr 5 19:24:45.775492000 (728h19m28.687829631s) [noise=true , address=81.171.14.70:4046 , piece_id=LMHJQTHJ7JYY35M3F7XWYVD274OKDBDKCSPTPVW75OD45I37WREQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 188.166.225.170:50152->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.775746100 (728h19m28.688083781s) [noise=true , address=81.171.14.70:4047 , piece_id=4GRNNNHVJEPJWTCB6GN65XBRL3TOOFNJEGSC3FNZOACHAFUAPOKA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection Apr 5 19:24:45.776021131 (728h19m28.688358762s) [noise=true , address=81.171.14.70:4031 , piece_id=JOJX7BWJCEIZQPIFFUSB7ZTDB6EY53JN4W4QSQGXTALSWTZYPVPA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection Apr 5 19:24:45.776035619 (728h19m28.68837329s) [noise=true , address=89.149.201.147:4002 , piece_id=L5JCOBLFBXEHYW54OA7WH7ZDH75VSJ2HJJDUXT64TGXWF3TNKSRQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc, address:89.149.201.147:4002): protocol: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection Apr 5 19:24:45.776457844 (728h19m28.688795475s) [noise=true , address=81.171.14.70:4041 , piece_id=U7R4MJPROP5OVX3ZNJS2QFUBBBWMH52G36ISP3QICJSUZ7VO5EDQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: read tcp 188.166.225.170:57676->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.776489063 (728h19m28.688826735s) [seg_pos={0 0}, upload=698890]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Apr 5 19:24:45.942762431 (728h19m28.855100052s) [upload=698890]: Done uploading segments... Apr 5 19:24:45.942795364 (728h19m28.855132995s) [upload=698890]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 4.530214241s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 4.585546368s trace: https://jaeger.dev.storj.io/trace/71e8b0fffa13491b FAIL (2026-04-05 20:06:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== 3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.445047209 (728h18m29.35738484s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.446660112 (728h18m29.358997743s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:47.230997446 (728h18m30.143335087s) [upload=698890]: Next returned nil segment Apr 5 19:23:47.231015220 (728h18m30.143352852s) [upload=698890]: Waiting for error group managing segments... Apr 5 19:23:47.620452706 (728h18m30.532790347s) [noise=true , address=185.167.204.205:4003 , piece_id=POAEFTZYX3ULC346GTXGT35G2C3CNGGKDBDH24MMLG3L5K6XEUQQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620924707 (728h18m30.533262338s) [noise=true , address=185.167.204.205:4006 , piece_id=OH4TLM4XGF4RFSIMJUCOZTRZAJ2VXSD5IZ272D5O4ZTRJV2EKXFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620995212 (728h18m30.533332843s) [noise=true , address=185.167.204.205:4036 , piece_id=VNGVFT6CPVAFI22JM7YP632ZHFHF63TYOS5OHMPN6SIJYI47Z6ZA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.623436224 (728h18m30.535773855s) [noise=true , address=185.167.204.206:4052 , piece_id=MURH7U3OF7FDWXLNFX2D62OLQXZBRHF5IH2B7O2NIDHYXYVMWPZQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.627891562 (728h18m30.540229193s) [noise=true , address=185.167.204.206:4064 , piece_id=R7LLF3HEHD6HVO4YOBXMNVUSXSOHZL4BDX2RSIL2SKGS3YLPJWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.628832688 (728h18m30.541170319s) [noise=true , address=185.167.204.206:4057 , piece_id=FVDAYDB2M3BS75XT6IWQRSP2QUEVFHSUF3EYAYHHQP37Z2OHS5XA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.629152575 (728h18m30.541490206s) [noise=true , address=185.167.204.206:4069 , piece_id=MWVVC2BOZMC4TWXPVELWHD4RZTBSMYR53CJSIPGQ25B7VDBTMQIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641386105 (728h18m30.553723736s) [noise=true , address=185.167.204.206:4043 , piece_id=VZKY2PYLTFYRGF2JGYWNSHRPEAX6GYTKW7ASDQL7ZALAXXXKNYFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641788261 (728h18m30.554125902s) [noise=true , address=185.167.204.205:4024 , piece_id=2N453JCGF4QS3LKGEZ7BDQZ45NRDPUTBZWECWUKOYBPBRLHJUD3Q, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.643005260 (728h18m30.555342891s) [noise=true , address=185.167.204.205:4011 , piece_id=BG66DKR26PWXMAFTAONWEUFPU6SMOOKMIZWNZSGOIRX2IMUJR4FQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.645249922 (728h18m30.557587553s) [noise=true , address=185.167.204.206:4065 , piece_id=EFD2RBNEIUL4XJ3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651359453 (728h18m30.563697084s) [noise=true , address=185.167.204.206:4056 , piece_id=J5L4SVTQZY7WSPSQ2GXOV7MDN4DIYE454N6FT6K7ZGUD7HR2BFHA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651412215 (728h18m30.563749846s) [noise=true , address=185.167.204.205:4034 , piece_id=VNTYO6Q4IMH2T2XEHJWH64F3YUL45BBCNQLZ46WUZAX62V2SMFTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.654801867 (728h18m30.567139498s) [noise=true , address=185.167.204.206:4071 , piece_id=5NSML2UJOWKUN3FUGXXD37NU3MARIKY6TDCLCDXC6S6WVFV7C5RA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.655101795 (728h18m30.567439426s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.658113638 (728h18m30.570451269s) [noise=true , address=185.167.204.206:4055 , piece_id=VIYGDMLV7V3V76NZ3V5IWBICO6CT7YNTP4BN5QHI542HNFVJ3PTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.662924132 (728h18m30.575261763s) [noise=true , address=185.167.204.206:4062 , piece_id=GRRUPYUEBCY2S6JPGJOEXNJ3OZZDZBYVX27AEMW5YV6IKGEZ6ZQA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.666865058 (728h18m30.579202689s) [noise=true , address=185.167.204.205:4033 , piece_id=EO2VKCTXXE7MA4UOPWABIV23LJDKGMKIUNV3KGFS47OIVXB5WVKQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.670168473 (728h18m30.582506104s) [noise=true , address=185.167.204.205:4009 , piece_id=YQ6P3DOBAPZKVCVTTBI2IJH2UMWWLMLWXMIJNRNSADDSHYF2KGAQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:51.907352601 (728h18m34.819690232s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:24:45.775054746 (728h19m28.687392447s) [noise=true , address=89.149.201.147:4011 , piece_id=FZK5PMW5EYIQLVOLCUOIWR2S42I7BLHX6FRV2MFOTPUKYF36GH2A, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection Apr 5 19:24:45.775168405 (728h19m28.687506076s) [noise=true , address=81.171.14.70:4048 , piece_id=MUDLJPWEYZYTMXLJJZ4FFOTGGVMXZVJSHXE5FZPNHE2SIW6LKPLA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 19:24:45.775267757 (728h19m28.687605408s) [noise=true , address=81.171.14.70:4045 , piece_id=3S6DJCM2FVYHVTFA64VBAWELYHHNKC6RZXBLCUUJS7XY3GACRPIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection Apr 5 19:24:45.775492000 (728h19m28.687829631s) [noise=true , address=81.171.14.70:4046 , piece_id=LMHJQTHJ7JYY35M3F7XWYVD274OKDBDKCSPTPVW75OD45I37WREQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 188.166.225.170:50152->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.775746100 (728h19m28.688083781s) [noise=true , address=81.171.14.70:4047 , piece_id=4GRNNNHVJEPJWTCB6GN65XBRL3TOOFNJEGSC3FNZOACHAFUAPOKA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection Apr 5 19:24:45.776021131 (728h19m28.688358762s) [noise=true , address=81.171.14.70:4031 , piece_id=JOJX7BWJCEIZQPIFFUSB7ZTDB6EY53JN4W4QSQGXTALSWTZYPVPA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection Apr 5 19:24:45.776035619 (728h19m28.68837329s) [noise=true , address=89.149.201.147:4002 , piece_id=L5JCOBLFBXEHYW54OA7WH7ZDH75VSJ2HJJDUXT64TGXWF3TNKSRQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc, address:89.149.201.147:4002): protocol: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection Apr 5 19:24:45.776457844 (728h19m28.688795475s) [noise=true , address=81.171.14.70:4041 , piece_id=U7R4MJPROP5OVX3ZNJS2QFUBBBWMH52G36ISP3QICJSUZ7VO5EDQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: read tcp 188.166.225.170:57676->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.776489063 (728h19m28.688826735s) [seg_pos={0 0}, upload=698890]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Apr 5 19:24:45.942762431 (728h19m28.855100052s) [upload=698890]: Done uploading segments... Apr 5 19:24:45.942795364 (728h19m28.855132995s) [upload=698890]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 4.585570786s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [X.X.........................................................] duration: 4.606576538s trace: https://jaeger.dev.storj.io/trace/71e8b0fffa13491b FAIL (2026-04-05 20:06:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== 3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.445047209 (728h18m29.35738484s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.446660112 (728h18m29.358997743s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:47.230997446 (728h18m30.143335087s) [upload=698890]: Next returned nil segment Apr 5 19:23:47.231015220 (728h18m30.143352852s) [upload=698890]: Waiting for error group managing segments... Apr 5 19:23:47.620452706 (728h18m30.532790347s) [noise=true , address=185.167.204.205:4003 , piece_id=POAEFTZYX3ULC346GTXGT35G2C3CNGGKDBDH24MMLG3L5K6XEUQQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620924707 (728h18m30.533262338s) [noise=true , address=185.167.204.205:4006 , piece_id=OH4TLM4XGF4RFSIMJUCOZTRZAJ2VXSD5IZ272D5O4ZTRJV2EKXFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620995212 (728h18m30.533332843s) [noise=true , address=185.167.204.205:4036 , piece_id=VNGVFT6CPVAFI22JM7YP632ZHFHF63TYOS5OHMPN6SIJYI47Z6ZA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.623436224 (728h18m30.535773855s) [noise=true , address=185.167.204.206:4052 , piece_id=MURH7U3OF7FDWXLNFX2D62OLQXZBRHF5IH2B7O2NIDHYXYVMWPZQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.627891562 (728h18m30.540229193s) [noise=true , address=185.167.204.206:4064 , piece_id=R7LLF3HEHD6HVO4YOBXMNVUSXSOHZL4BDX2RSIL2SKGS3YLPJWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.628832688 (728h18m30.541170319s) [noise=true , address=185.167.204.206:4057 , piece_id=FVDAYDB2M3BS75XT6IWQRSP2QUEVFHSUF3EYAYHHQP37Z2OHS5XA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.629152575 (728h18m30.541490206s) [noise=true , address=185.167.204.206:4069 , piece_id=MWVVC2BOZMC4TWXPVELWHD4RZTBSMYR53CJSIPGQ25B7VDBTMQIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641386105 (728h18m30.553723736s) [noise=true , address=185.167.204.206:4043 , piece_id=VZKY2PYLTFYRGF2JGYWNSHRPEAX6GYTKW7ASDQL7ZALAXXXKNYFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641788261 (728h18m30.554125902s) [noise=true , address=185.167.204.205:4024 , piece_id=2N453JCGF4QS3LKGEZ7BDQZ45NRDPUTBZWECWUKOYBPBRLHJUD3Q, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.643005260 (728h18m30.555342891s) [noise=true , address=185.167.204.205:4011 , piece_id=BG66DKR26PWXMAFTAONWEUFPU6SMOOKMIZWNZSGOIRX2IMUJR4FQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.645249922 (728h18m30.557587553s) [noise=true , address=185.167.204.206:4065 , piece_id=EFD2RBNEIUL4XJ3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651359453 (728h18m30.563697084s) [noise=true , address=185.167.204.206:4056 , piece_id=J5L4SVTQZY7WSPSQ2GXOV7MDN4DIYE454N6FT6K7ZGUD7HR2BFHA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651412215 (728h18m30.563749846s) [noise=true , address=185.167.204.205:4034 , piece_id=VNTYO6Q4IMH2T2XEHJWH64F3YUL45BBCNQLZ46WUZAX62V2SMFTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.654801867 (728h18m30.567139498s) [noise=true , address=185.167.204.206:4071 , piece_id=5NSML2UJOWKUN3FUGXXD37NU3MARIKY6TDCLCDXC6S6WVFV7C5RA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.655101795 (728h18m30.567439426s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.658113638 (728h18m30.570451269s) [noise=true , address=185.167.204.206:4055 , piece_id=VIYGDMLV7V3V76NZ3V5IWBICO6CT7YNTP4BN5QHI542HNFVJ3PTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.662924132 (728h18m30.575261763s) [noise=true , address=185.167.204.206:4062 , piece_id=GRRUPYUEBCY2S6JPGJOEXNJ3OZZDZBYVX27AEMW5YV6IKGEZ6ZQA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.666865058 (728h18m30.579202689s) [noise=true , address=185.167.204.205:4033 , piece_id=EO2VKCTXXE7MA4UOPWABIV23LJDKGMKIUNV3KGFS47OIVXB5WVKQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.670168473 (728h18m30.582506104s) [noise=true , address=185.167.204.205:4009 , piece_id=YQ6P3DOBAPZKVCVTTBI2IJH2UMWWLMLWXMIJNRNSADDSHYF2KGAQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:51.907352601 (728h18m34.819690232s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:24:45.775054746 (728h19m28.687392447s) [noise=true , address=89.149.201.147:4011 , piece_id=FZK5PMW5EYIQLVOLCUOIWR2S42I7BLHX6FRV2MFOTPUKYF36GH2A, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection Apr 5 19:24:45.775168405 (728h19m28.687506076s) [noise=true , address=81.171.14.70:4048 , piece_id=MUDLJPWEYZYTMXLJJZ4FFOTGGVMXZVJSHXE5FZPNHE2SIW6LKPLA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 19:24:45.775267757 (728h19m28.687605408s) [noise=true , address=81.171.14.70:4045 , piece_id=3S6DJCM2FVYHVTFA64VBAWELYHHNKC6RZXBLCUUJS7XY3GACRPIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection Apr 5 19:24:45.775492000 (728h19m28.687829631s) [noise=true , address=81.171.14.70:4046 , piece_id=LMHJQTHJ7JYY35M3F7XWYVD274OKDBDKCSPTPVW75OD45I37WREQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 188.166.225.170:50152->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.775746100 (728h19m28.688083781s) [noise=true , address=81.171.14.70:4047 , piece_id=4GRNNNHVJEPJWTCB6GN65XBRL3TOOFNJEGSC3FNZOACHAFUAPOKA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection Apr 5 19:24:45.776021131 (728h19m28.688358762s) [noise=true , address=81.171.14.70:4031 , piece_id=JOJX7BWJCEIZQPIFFUSB7ZTDB6EY53JN4W4QSQGXTALSWTZYPVPA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection Apr 5 19:24:45.776035619 (728h19m28.68837329s) [noise=true , address=89.149.201.147:4002 , piece_id=L5JCOBLFBXEHYW54OA7WH7ZDH75VSJ2HJJDUXT64TGXWF3TNKSRQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc, address:89.149.201.147:4002): protocol: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection Apr 5 19:24:45.776457844 (728h19m28.688795475s) [noise=true , address=81.171.14.70:4041 , piece_id=U7R4MJPROP5OVX3ZNJS2QFUBBBWMH52G36ISP3QICJSUZ7VO5EDQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: read tcp 188.166.225.170:57676->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.776489063 (728h19m28.688826735s) [seg_pos={0 0}, upload=698890]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Apr 5 19:24:45.942762431 (728h19m28.855100052s) [upload=698890]: Done uploading segments... Apr 5 19:24:45.942795364 (728h19m28.855132995s) [upload=698890]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 4.606602689s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.X.X........................................................] duration: 4.522775613s trace: https://jaeger.dev.storj.io/trace/71e8b0fffa13491b FAIL (2026-04-05 20:05:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== 3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.445047209 (728h18m29.35738484s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.446660112 (728h18m29.358997743s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:47.230997446 (728h18m30.143335087s) [upload=698890]: Next returned nil segment Apr 5 19:23:47.231015220 (728h18m30.143352852s) [upload=698890]: Waiting for error group managing segments... Apr 5 19:23:47.620452706 (728h18m30.532790347s) [noise=true , address=185.167.204.205:4003 , piece_id=POAEFTZYX3ULC346GTXGT35G2C3CNGGKDBDH24MMLG3L5K6XEUQQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620924707 (728h18m30.533262338s) [noise=true , address=185.167.204.205:4006 , piece_id=OH4TLM4XGF4RFSIMJUCOZTRZAJ2VXSD5IZ272D5O4ZTRJV2EKXFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620995212 (728h18m30.533332843s) [noise=true , address=185.167.204.205:4036 , piece_id=VNGVFT6CPVAFI22JM7YP632ZHFHF63TYOS5OHMPN6SIJYI47Z6ZA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.623436224 (728h18m30.535773855s) [noise=true , address=185.167.204.206:4052 , piece_id=MURH7U3OF7FDWXLNFX2D62OLQXZBRHF5IH2B7O2NIDHYXYVMWPZQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.627891562 (728h18m30.540229193s) [noise=true , address=185.167.204.206:4064 , piece_id=R7LLF3HEHD6HVO4YOBXMNVUSXSOHZL4BDX2RSIL2SKGS3YLPJWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.628832688 (728h18m30.541170319s) [noise=true , address=185.167.204.206:4057 , piece_id=FVDAYDB2M3BS75XT6IWQRSP2QUEVFHSUF3EYAYHHQP37Z2OHS5XA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.629152575 (728h18m30.541490206s) [noise=true , address=185.167.204.206:4069 , piece_id=MWVVC2BOZMC4TWXPVELWHD4RZTBSMYR53CJSIPGQ25B7VDBTMQIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641386105 (728h18m30.553723736s) [noise=true , address=185.167.204.206:4043 , piece_id=VZKY2PYLTFYRGF2JGYWNSHRPEAX6GYTKW7ASDQL7ZALAXXXKNYFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641788261 (728h18m30.554125902s) [noise=true , address=185.167.204.205:4024 , piece_id=2N453JCGF4QS3LKGEZ7BDQZ45NRDPUTBZWECWUKOYBPBRLHJUD3Q, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.643005260 (728h18m30.555342891s) [noise=true , address=185.167.204.205:4011 , piece_id=BG66DKR26PWXMAFTAONWEUFPU6SMOOKMIZWNZSGOIRX2IMUJR4FQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.645249922 (728h18m30.557587553s) [noise=true , address=185.167.204.206:4065 , piece_id=EFD2RBNEIUL4XJ3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651359453 (728h18m30.563697084s) [noise=true , address=185.167.204.206:4056 , piece_id=J5L4SVTQZY7WSPSQ2GXOV7MDN4DIYE454N6FT6K7ZGUD7HR2BFHA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651412215 (728h18m30.563749846s) [noise=true , address=185.167.204.205:4034 , piece_id=VNTYO6Q4IMH2T2XEHJWH64F3YUL45BBCNQLZ46WUZAX62V2SMFTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.654801867 (728h18m30.567139498s) [noise=true , address=185.167.204.206:4071 , piece_id=5NSML2UJOWKUN3FUGXXD37NU3MARIKY6TDCLCDXC6S6WVFV7C5RA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.655101795 (728h18m30.567439426s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.658113638 (728h18m30.570451269s) [noise=true , address=185.167.204.206:4055 , piece_id=VIYGDMLV7V3V76NZ3V5IWBICO6CT7YNTP4BN5QHI542HNFVJ3PTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.662924132 (728h18m30.575261763s) [noise=true , address=185.167.204.206:4062 , piece_id=GRRUPYUEBCY2S6JPGJOEXNJ3OZZDZBYVX27AEMW5YV6IKGEZ6ZQA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.666865058 (728h18m30.579202689s) [noise=true , address=185.167.204.205:4033 , piece_id=EO2VKCTXXE7MA4UOPWABIV23LJDKGMKIUNV3KGFS47OIVXB5WVKQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.670168473 (728h18m30.582506104s) [noise=true , address=185.167.204.205:4009 , piece_id=YQ6P3DOBAPZKVCVTTBI2IJH2UMWWLMLWXMIJNRNSADDSHYF2KGAQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:51.907352601 (728h18m34.819690232s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:24:45.775054746 (728h19m28.687392447s) [noise=true , address=89.149.201.147:4011 , piece_id=FZK5PMW5EYIQLVOLCUOIWR2S42I7BLHX6FRV2MFOTPUKYF36GH2A, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection Apr 5 19:24:45.775168405 (728h19m28.687506076s) [noise=true , address=81.171.14.70:4048 , piece_id=MUDLJPWEYZYTMXLJJZ4FFOTGGVMXZVJSHXE5FZPNHE2SIW6LKPLA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 19:24:45.775267757 (728h19m28.687605408s) [noise=true , address=81.171.14.70:4045 , piece_id=3S6DJCM2FVYHVTFA64VBAWELYHHNKC6RZXBLCUUJS7XY3GACRPIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection Apr 5 19:24:45.775492000 (728h19m28.687829631s) [noise=true , address=81.171.14.70:4046 , piece_id=LMHJQTHJ7JYY35M3F7XWYVD274OKDBDKCSPTPVW75OD45I37WREQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 188.166.225.170:50152->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.775746100 (728h19m28.688083781s) [noise=true , address=81.171.14.70:4047 , piece_id=4GRNNNHVJEPJWTCB6GN65XBRL3TOOFNJEGSC3FNZOACHAFUAPOKA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection Apr 5 19:24:45.776021131 (728h19m28.688358762s) [noise=true , address=81.171.14.70:4031 , piece_id=JOJX7BWJCEIZQPIFFUSB7ZTDB6EY53JN4W4QSQGXTALSWTZYPVPA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection Apr 5 19:24:45.776035619 (728h19m28.68837329s) [noise=true , address=89.149.201.147:4002 , piece_id=L5JCOBLFBXEHYW54OA7WH7ZDH75VSJ2HJJDUXT64TGXWF3TNKSRQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc, address:89.149.201.147:4002): protocol: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection Apr 5 19:24:45.776457844 (728h19m28.688795475s) [noise=true , address=81.171.14.70:4041 , piece_id=U7R4MJPROP5OVX3ZNJS2QFUBBBWMH52G36ISP3QICJSUZ7VO5EDQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: read tcp 188.166.225.170:57676->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.776489063 (728h19m28.688826735s) [seg_pos={0 0}, upload=698890]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Apr 5 19:24:45.942762431 (728h19m28.855100052s) [upload=698890]: Done uploading segments... Apr 5 19:24:45.942795364 (728h19m28.855132995s) [upload=698890]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 4.522793729s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..X.X.......................................................] duration: 4.503922943s trace: https://jaeger.dev.storj.io/trace/71e8b0fffa13491b FAIL (2026-04-05 20:05:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== 3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.445047209 (728h18m29.35738484s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.446660112 (728h18m29.358997743s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:47.230997446 (728h18m30.143335087s) [upload=698890]: Next returned nil segment Apr 5 19:23:47.231015220 (728h18m30.143352852s) [upload=698890]: Waiting for error group managing segments... Apr 5 19:23:47.620452706 (728h18m30.532790347s) [noise=true , address=185.167.204.205:4003 , piece_id=POAEFTZYX3ULC346GTXGT35G2C3CNGGKDBDH24MMLG3L5K6XEUQQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620924707 (728h18m30.533262338s) [noise=true , address=185.167.204.205:4006 , piece_id=OH4TLM4XGF4RFSIMJUCOZTRZAJ2VXSD5IZ272D5O4ZTRJV2EKXFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620995212 (728h18m30.533332843s) [noise=true , address=185.167.204.205:4036 , piece_id=VNGVFT6CPVAFI22JM7YP632ZHFHF63TYOS5OHMPN6SIJYI47Z6ZA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.623436224 (728h18m30.535773855s) [noise=true , address=185.167.204.206:4052 , piece_id=MURH7U3OF7FDWXLNFX2D62OLQXZBRHF5IH2B7O2NIDHYXYVMWPZQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.627891562 (728h18m30.540229193s) [noise=true , address=185.167.204.206:4064 , piece_id=R7LLF3HEHD6HVO4YOBXMNVUSXSOHZL4BDX2RSIL2SKGS3YLPJWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.628832688 (728h18m30.541170319s) [noise=true , address=185.167.204.206:4057 , piece_id=FVDAYDB2M3BS75XT6IWQRSP2QUEVFHSUF3EYAYHHQP37Z2OHS5XA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.629152575 (728h18m30.541490206s) [noise=true , address=185.167.204.206:4069 , piece_id=MWVVC2BOZMC4TWXPVELWHD4RZTBSMYR53CJSIPGQ25B7VDBTMQIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641386105 (728h18m30.553723736s) [noise=true , address=185.167.204.206:4043 , piece_id=VZKY2PYLTFYRGF2JGYWNSHRPEAX6GYTKW7ASDQL7ZALAXXXKNYFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641788261 (728h18m30.554125902s) [noise=true , address=185.167.204.205:4024 , piece_id=2N453JCGF4QS3LKGEZ7BDQZ45NRDPUTBZWECWUKOYBPBRLHJUD3Q, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.643005260 (728h18m30.555342891s) [noise=true , address=185.167.204.205:4011 , piece_id=BG66DKR26PWXMAFTAONWEUFPU6SMOOKMIZWNZSGOIRX2IMUJR4FQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.645249922 (728h18m30.557587553s) [noise=true , address=185.167.204.206:4065 , piece_id=EFD2RBNEIUL4XJ3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651359453 (728h18m30.563697084s) [noise=true , address=185.167.204.206:4056 , piece_id=J5L4SVTQZY7WSPSQ2GXOV7MDN4DIYE454N6FT6K7ZGUD7HR2BFHA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651412215 (728h18m30.563749846s) [noise=true , address=185.167.204.205:4034 , piece_id=VNTYO6Q4IMH2T2XEHJWH64F3YUL45BBCNQLZ46WUZAX62V2SMFTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.654801867 (728h18m30.567139498s) [noise=true , address=185.167.204.206:4071 , piece_id=5NSML2UJOWKUN3FUGXXD37NU3MARIKY6TDCLCDXC6S6WVFV7C5RA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.655101795 (728h18m30.567439426s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.658113638 (728h18m30.570451269s) [noise=true , address=185.167.204.206:4055 , piece_id=VIYGDMLV7V3V76NZ3V5IWBICO6CT7YNTP4BN5QHI542HNFVJ3PTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.662924132 (728h18m30.575261763s) [noise=true , address=185.167.204.206:4062 , piece_id=GRRUPYUEBCY2S6JPGJOEXNJ3OZZDZBYVX27AEMW5YV6IKGEZ6ZQA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.666865058 (728h18m30.579202689s) [noise=true , address=185.167.204.205:4033 , piece_id=EO2VKCTXXE7MA4UOPWABIV23LJDKGMKIUNV3KGFS47OIVXB5WVKQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.670168473 (728h18m30.582506104s) [noise=true , address=185.167.204.205:4009 , piece_id=YQ6P3DOBAPZKVCVTTBI2IJH2UMWWLMLWXMIJNRNSADDSHYF2KGAQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:51.907352601 (728h18m34.819690232s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:24:45.775054746 (728h19m28.687392447s) [noise=true , address=89.149.201.147:4011 , piece_id=FZK5PMW5EYIQLVOLCUOIWR2S42I7BLHX6FRV2MFOTPUKYF36GH2A, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection Apr 5 19:24:45.775168405 (728h19m28.687506076s) [noise=true , address=81.171.14.70:4048 , piece_id=MUDLJPWEYZYTMXLJJZ4FFOTGGVMXZVJSHXE5FZPNHE2SIW6LKPLA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 19:24:45.775267757 (728h19m28.687605408s) [noise=true , address=81.171.14.70:4045 , piece_id=3S6DJCM2FVYHVTFA64VBAWELYHHNKC6RZXBLCUUJS7XY3GACRPIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection Apr 5 19:24:45.775492000 (728h19m28.687829631s) [noise=true , address=81.171.14.70:4046 , piece_id=LMHJQTHJ7JYY35M3F7XWYVD274OKDBDKCSPTPVW75OD45I37WREQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 188.166.225.170:50152->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.775746100 (728h19m28.688083781s) [noise=true , address=81.171.14.70:4047 , piece_id=4GRNNNHVJEPJWTCB6GN65XBRL3TOOFNJEGSC3FNZOACHAFUAPOKA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection Apr 5 19:24:45.776021131 (728h19m28.688358762s) [noise=true , address=81.171.14.70:4031 , piece_id=JOJX7BWJCEIZQPIFFUSB7ZTDB6EY53JN4W4QSQGXTALSWTZYPVPA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection Apr 5 19:24:45.776035619 (728h19m28.68837329s) [noise=true , address=89.149.201.147:4002 , piece_id=L5JCOBLFBXEHYW54OA7WH7ZDH75VSJ2HJJDUXT64TGXWF3TNKSRQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc, address:89.149.201.147:4002): protocol: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection Apr 5 19:24:45.776457844 (728h19m28.688795475s) [noise=true , address=81.171.14.70:4041 , piece_id=U7R4MJPROP5OVX3ZNJS2QFUBBBWMH52G36ISP3QICJSUZ7VO5EDQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: read tcp 188.166.225.170:57676->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.776489063 (728h19m28.688826735s) [seg_pos={0 0}, upload=698890]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Apr 5 19:24:45.942762431 (728h19m28.855100052s) [upload=698890]: Done uploading segments... Apr 5 19:24:45.942795364 (728h19m28.855132995s) [upload=698890]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 4.503947992s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...X.X......................................................] duration: 4.600919494s trace: https://jaeger.dev.storj.io/trace/71e8b0fffa13491b FAIL (2026-04-05 20:05:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== 3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.445047209 (728h18m29.35738484s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.446660112 (728h18m29.358997743s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:47.230997446 (728h18m30.143335087s) [upload=698890]: Next returned nil segment Apr 5 19:23:47.231015220 (728h18m30.143352852s) [upload=698890]: Waiting for error group managing segments... Apr 5 19:23:47.620452706 (728h18m30.532790347s) [noise=true , address=185.167.204.205:4003 , piece_id=POAEFTZYX3ULC346GTXGT35G2C3CNGGKDBDH24MMLG3L5K6XEUQQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620924707 (728h18m30.533262338s) [noise=true , address=185.167.204.205:4006 , piece_id=OH4TLM4XGF4RFSIMJUCOZTRZAJ2VXSD5IZ272D5O4ZTRJV2EKXFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620995212 (728h18m30.533332843s) [noise=true , address=185.167.204.205:4036 , piece_id=VNGVFT6CPVAFI22JM7YP632ZHFHF63TYOS5OHMPN6SIJYI47Z6ZA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.623436224 (728h18m30.535773855s) [noise=true , address=185.167.204.206:4052 , piece_id=MURH7U3OF7FDWXLNFX2D62OLQXZBRHF5IH2B7O2NIDHYXYVMWPZQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.627891562 (728h18m30.540229193s) [noise=true , address=185.167.204.206:4064 , piece_id=R7LLF3HEHD6HVO4YOBXMNVUSXSOHZL4BDX2RSIL2SKGS3YLPJWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.628832688 (728h18m30.541170319s) [noise=true , address=185.167.204.206:4057 , piece_id=FVDAYDB2M3BS75XT6IWQRSP2QUEVFHSUF3EYAYHHQP37Z2OHS5XA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.629152575 (728h18m30.541490206s) [noise=true , address=185.167.204.206:4069 , piece_id=MWVVC2BOZMC4TWXPVELWHD4RZTBSMYR53CJSIPGQ25B7VDBTMQIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641386105 (728h18m30.553723736s) [noise=true , address=185.167.204.206:4043 , piece_id=VZKY2PYLTFYRGF2JGYWNSHRPEAX6GYTKW7ASDQL7ZALAXXXKNYFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641788261 (728h18m30.554125902s) [noise=true , address=185.167.204.205:4024 , piece_id=2N453JCGF4QS3LKGEZ7BDQZ45NRDPUTBZWECWUKOYBPBRLHJUD3Q, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.643005260 (728h18m30.555342891s) [noise=true , address=185.167.204.205:4011 , piece_id=BG66DKR26PWXMAFTAONWEUFPU6SMOOKMIZWNZSGOIRX2IMUJR4FQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.645249922 (728h18m30.557587553s) [noise=true , address=185.167.204.206:4065 , piece_id=EFD2RBNEIUL4XJ3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651359453 (728h18m30.563697084s) [noise=true , address=185.167.204.206:4056 , piece_id=J5L4SVTQZY7WSPSQ2GXOV7MDN4DIYE454N6FT6K7ZGUD7HR2BFHA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651412215 (728h18m30.563749846s) [noise=true , address=185.167.204.205:4034 , piece_id=VNTYO6Q4IMH2T2XEHJWH64F3YUL45BBCNQLZ46WUZAX62V2SMFTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.654801867 (728h18m30.567139498s) [noise=true , address=185.167.204.206:4071 , piece_id=5NSML2UJOWKUN3FUGXXD37NU3MARIKY6TDCLCDXC6S6WVFV7C5RA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.655101795 (728h18m30.567439426s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.658113638 (728h18m30.570451269s) [noise=true , address=185.167.204.206:4055 , piece_id=VIYGDMLV7V3V76NZ3V5IWBICO6CT7YNTP4BN5QHI542HNFVJ3PTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.662924132 (728h18m30.575261763s) [noise=true , address=185.167.204.206:4062 , piece_id=GRRUPYUEBCY2S6JPGJOEXNJ3OZZDZBYVX27AEMW5YV6IKGEZ6ZQA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.666865058 (728h18m30.579202689s) [noise=true , address=185.167.204.205:4033 , piece_id=EO2VKCTXXE7MA4UOPWABIV23LJDKGMKIUNV3KGFS47OIVXB5WVKQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.670168473 (728h18m30.582506104s) [noise=true , address=185.167.204.205:4009 , piece_id=YQ6P3DOBAPZKVCVTTBI2IJH2UMWWLMLWXMIJNRNSADDSHYF2KGAQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:51.907352601 (728h18m34.819690232s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:24:45.775054746 (728h19m28.687392447s) [noise=true , address=89.149.201.147:4011 , piece_id=FZK5PMW5EYIQLVOLCUOIWR2S42I7BLHX6FRV2MFOTPUKYF36GH2A, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection Apr 5 19:24:45.775168405 (728h19m28.687506076s) [noise=true , address=81.171.14.70:4048 , piece_id=MUDLJPWEYZYTMXLJJZ4FFOTGGVMXZVJSHXE5FZPNHE2SIW6LKPLA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 19:24:45.775267757 (728h19m28.687605408s) [noise=true , address=81.171.14.70:4045 , piece_id=3S6DJCM2FVYHVTFA64VBAWELYHHNKC6RZXBLCUUJS7XY3GACRPIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection Apr 5 19:24:45.775492000 (728h19m28.687829631s) [noise=true , address=81.171.14.70:4046 , piece_id=LMHJQTHJ7JYY35M3F7XWYVD274OKDBDKCSPTPVW75OD45I37WREQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 188.166.225.170:50152->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.775746100 (728h19m28.688083781s) [noise=true , address=81.171.14.70:4047 , piece_id=4GRNNNHVJEPJWTCB6GN65XBRL3TOOFNJEGSC3FNZOACHAFUAPOKA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection Apr 5 19:24:45.776021131 (728h19m28.688358762s) [noise=true , address=81.171.14.70:4031 , piece_id=JOJX7BWJCEIZQPIFFUSB7ZTDB6EY53JN4W4QSQGXTALSWTZYPVPA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection Apr 5 19:24:45.776035619 (728h19m28.68837329s) [noise=true , address=89.149.201.147:4002 , piece_id=L5JCOBLFBXEHYW54OA7WH7ZDH75VSJ2HJJDUXT64TGXWF3TNKSRQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc, address:89.149.201.147:4002): protocol: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection Apr 5 19:24:45.776457844 (728h19m28.688795475s) [noise=true , address=81.171.14.70:4041 , piece_id=U7R4MJPROP5OVX3ZNJS2QFUBBBWMH52G36ISP3QICJSUZ7VO5EDQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: read tcp 188.166.225.170:57676->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.776489063 (728h19m28.688826735s) [seg_pos={0 0}, upload=698890]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Apr 5 19:24:45.942762431 (728h19m28.855100052s) [upload=698890]: Done uploading segments... Apr 5 19:24:45.942795364 (728h19m28.855132995s) [upload=698890]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 4.600953359s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [....X.X.....................................................] duration: 4.664088538s trace: https://jaeger.dev.storj.io/trace/71e8b0fffa13491b FAIL (2026-04-05 20:04:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== 3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.445047209 (728h18m29.35738484s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.446660112 (728h18m29.358997743s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:47.230997446 (728h18m30.143335087s) [upload=698890]: Next returned nil segment Apr 5 19:23:47.231015220 (728h18m30.143352852s) [upload=698890]: Waiting for error group managing segments... Apr 5 19:23:47.620452706 (728h18m30.532790347s) [noise=true , address=185.167.204.205:4003 , piece_id=POAEFTZYX3ULC346GTXGT35G2C3CNGGKDBDH24MMLG3L5K6XEUQQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620924707 (728h18m30.533262338s) [noise=true , address=185.167.204.205:4006 , piece_id=OH4TLM4XGF4RFSIMJUCOZTRZAJ2VXSD5IZ272D5O4ZTRJV2EKXFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620995212 (728h18m30.533332843s) [noise=true , address=185.167.204.205:4036 , piece_id=VNGVFT6CPVAFI22JM7YP632ZHFHF63TYOS5OHMPN6SIJYI47Z6ZA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.623436224 (728h18m30.535773855s) [noise=true , address=185.167.204.206:4052 , piece_id=MURH7U3OF7FDWXLNFX2D62OLQXZBRHF5IH2B7O2NIDHYXYVMWPZQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.627891562 (728h18m30.540229193s) [noise=true , address=185.167.204.206:4064 , piece_id=R7LLF3HEHD6HVO4YOBXMNVUSXSOHZL4BDX2RSIL2SKGS3YLPJWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.628832688 (728h18m30.541170319s) [noise=true , address=185.167.204.206:4057 , piece_id=FVDAYDB2M3BS75XT6IWQRSP2QUEVFHSUF3EYAYHHQP37Z2OHS5XA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.629152575 (728h18m30.541490206s) [noise=true , address=185.167.204.206:4069 , piece_id=MWVVC2BOZMC4TWXPVELWHD4RZTBSMYR53CJSIPGQ25B7VDBTMQIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641386105 (728h18m30.553723736s) [noise=true , address=185.167.204.206:4043 , piece_id=VZKY2PYLTFYRGF2JGYWNSHRPEAX6GYTKW7ASDQL7ZALAXXXKNYFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641788261 (728h18m30.554125902s) [noise=true , address=185.167.204.205:4024 , piece_id=2N453JCGF4QS3LKGEZ7BDQZ45NRDPUTBZWECWUKOYBPBRLHJUD3Q, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.643005260 (728h18m30.555342891s) [noise=true , address=185.167.204.205:4011 , piece_id=BG66DKR26PWXMAFTAONWEUFPU6SMOOKMIZWNZSGOIRX2IMUJR4FQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.645249922 (728h18m30.557587553s) [noise=true , address=185.167.204.206:4065 , piece_id=EFD2RBNEIUL4XJ3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651359453 (728h18m30.563697084s) [noise=true , address=185.167.204.206:4056 , piece_id=J5L4SVTQZY7WSPSQ2GXOV7MDN4DIYE454N6FT6K7ZGUD7HR2BFHA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651412215 (728h18m30.563749846s) [noise=true , address=185.167.204.205:4034 , piece_id=VNTYO6Q4IMH2T2XEHJWH64F3YUL45BBCNQLZ46WUZAX62V2SMFTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.654801867 (728h18m30.567139498s) [noise=true , address=185.167.204.206:4071 , piece_id=5NSML2UJOWKUN3FUGXXD37NU3MARIKY6TDCLCDXC6S6WVFV7C5RA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.655101795 (728h18m30.567439426s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.658113638 (728h18m30.570451269s) [noise=true , address=185.167.204.206:4055 , piece_id=VIYGDMLV7V3V76NZ3V5IWBICO6CT7YNTP4BN5QHI542HNFVJ3PTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.662924132 (728h18m30.575261763s) [noise=true , address=185.167.204.206:4062 , piece_id=GRRUPYUEBCY2S6JPGJOEXNJ3OZZDZBYVX27AEMW5YV6IKGEZ6ZQA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.666865058 (728h18m30.579202689s) [noise=true , address=185.167.204.205:4033 , piece_id=EO2VKCTXXE7MA4UOPWABIV23LJDKGMKIUNV3KGFS47OIVXB5WVKQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.670168473 (728h18m30.582506104s) [noise=true , address=185.167.204.205:4009 , piece_id=YQ6P3DOBAPZKVCVTTBI2IJH2UMWWLMLWXMIJNRNSADDSHYF2KGAQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:51.907352601 (728h18m34.819690232s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:24:45.775054746 (728h19m28.687392447s) [noise=true , address=89.149.201.147:4011 , piece_id=FZK5PMW5EYIQLVOLCUOIWR2S42I7BLHX6FRV2MFOTPUKYF36GH2A, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection Apr 5 19:24:45.775168405 (728h19m28.687506076s) [noise=true , address=81.171.14.70:4048 , piece_id=MUDLJPWEYZYTMXLJJZ4FFOTGGVMXZVJSHXE5FZPNHE2SIW6LKPLA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 19:24:45.775267757 (728h19m28.687605408s) [noise=true , address=81.171.14.70:4045 , piece_id=3S6DJCM2FVYHVTFA64VBAWELYHHNKC6RZXBLCUUJS7XY3GACRPIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection Apr 5 19:24:45.775492000 (728h19m28.687829631s) [noise=true , address=81.171.14.70:4046 , piece_id=LMHJQTHJ7JYY35M3F7XWYVD274OKDBDKCSPTPVW75OD45I37WREQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 188.166.225.170:50152->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.775746100 (728h19m28.688083781s) [noise=true , address=81.171.14.70:4047 , piece_id=4GRNNNHVJEPJWTCB6GN65XBRL3TOOFNJEGSC3FNZOACHAFUAPOKA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection Apr 5 19:24:45.776021131 (728h19m28.688358762s) [noise=true , address=81.171.14.70:4031 , piece_id=JOJX7BWJCEIZQPIFFUSB7ZTDB6EY53JN4W4QSQGXTALSWTZYPVPA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection Apr 5 19:24:45.776035619 (728h19m28.68837329s) [noise=true , address=89.149.201.147:4002 , piece_id=L5JCOBLFBXEHYW54OA7WH7ZDH75VSJ2HJJDUXT64TGXWF3TNKSRQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc, address:89.149.201.147:4002): protocol: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection Apr 5 19:24:45.776457844 (728h19m28.688795475s) [noise=true , address=81.171.14.70:4041 , piece_id=U7R4MJPROP5OVX3ZNJS2QFUBBBWMH52G36ISP3QICJSUZ7VO5EDQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: read tcp 188.166.225.170:57676->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.776489063 (728h19m28.688826735s) [seg_pos={0 0}, upload=698890]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Apr 5 19:24:45.942762431 (728h19m28.855100052s) [upload=698890]: Done uploading segments... Apr 5 19:24:45.942795364 (728h19m28.855132995s) [upload=698890]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 4.664114538s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.....X.X....................................................] duration: 4.644074591s trace: https://jaeger.dev.storj.io/trace/71e8b0fffa13491b FAIL (2026-04-05 20:03:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== 3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.445047209 (728h18m29.35738484s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.446660112 (728h18m29.358997743s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:47.230997446 (728h18m30.143335087s) [upload=698890]: Next returned nil segment Apr 5 19:23:47.231015220 (728h18m30.143352852s) [upload=698890]: Waiting for error group managing segments... Apr 5 19:23:47.620452706 (728h18m30.532790347s) [noise=true , address=185.167.204.205:4003 , piece_id=POAEFTZYX3ULC346GTXGT35G2C3CNGGKDBDH24MMLG3L5K6XEUQQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620924707 (728h18m30.533262338s) [noise=true , address=185.167.204.205:4006 , piece_id=OH4TLM4XGF4RFSIMJUCOZTRZAJ2VXSD5IZ272D5O4ZTRJV2EKXFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620995212 (728h18m30.533332843s) [noise=true , address=185.167.204.205:4036 , piece_id=VNGVFT6CPVAFI22JM7YP632ZHFHF63TYOS5OHMPN6SIJYI47Z6ZA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.623436224 (728h18m30.535773855s) [noise=true , address=185.167.204.206:4052 , piece_id=MURH7U3OF7FDWXLNFX2D62OLQXZBRHF5IH2B7O2NIDHYXYVMWPZQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.627891562 (728h18m30.540229193s) [noise=true , address=185.167.204.206:4064 , piece_id=R7LLF3HEHD6HVO4YOBXMNVUSXSOHZL4BDX2RSIL2SKGS3YLPJWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.628832688 (728h18m30.541170319s) [noise=true , address=185.167.204.206:4057 , piece_id=FVDAYDB2M3BS75XT6IWQRSP2QUEVFHSUF3EYAYHHQP37Z2OHS5XA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.629152575 (728h18m30.541490206s) [noise=true , address=185.167.204.206:4069 , piece_id=MWVVC2BOZMC4TWXPVELWHD4RZTBSMYR53CJSIPGQ25B7VDBTMQIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641386105 (728h18m30.553723736s) [noise=true , address=185.167.204.206:4043 , piece_id=VZKY2PYLTFYRGF2JGYWNSHRPEAX6GYTKW7ASDQL7ZALAXXXKNYFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641788261 (728h18m30.554125902s) [noise=true , address=185.167.204.205:4024 , piece_id=2N453JCGF4QS3LKGEZ7BDQZ45NRDPUTBZWECWUKOYBPBRLHJUD3Q, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.643005260 (728h18m30.555342891s) [noise=true , address=185.167.204.205:4011 , piece_id=BG66DKR26PWXMAFTAONWEUFPU6SMOOKMIZWNZSGOIRX2IMUJR4FQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.645249922 (728h18m30.557587553s) [noise=true , address=185.167.204.206:4065 , piece_id=EFD2RBNEIUL4XJ3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651359453 (728h18m30.563697084s) [noise=true , address=185.167.204.206:4056 , piece_id=J5L4SVTQZY7WSPSQ2GXOV7MDN4DIYE454N6FT6K7ZGUD7HR2BFHA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651412215 (728h18m30.563749846s) [noise=true , address=185.167.204.205:4034 , piece_id=VNTYO6Q4IMH2T2XEHJWH64F3YUL45BBCNQLZ46WUZAX62V2SMFTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.654801867 (728h18m30.567139498s) [noise=true , address=185.167.204.206:4071 , piece_id=5NSML2UJOWKUN3FUGXXD37NU3MARIKY6TDCLCDXC6S6WVFV7C5RA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.655101795 (728h18m30.567439426s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.658113638 (728h18m30.570451269s) [noise=true , address=185.167.204.206:4055 , piece_id=VIYGDMLV7V3V76NZ3V5IWBICO6CT7YNTP4BN5QHI542HNFVJ3PTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.662924132 (728h18m30.575261763s) [noise=true , address=185.167.204.206:4062 , piece_id=GRRUPYUEBCY2S6JPGJOEXNJ3OZZDZBYVX27AEMW5YV6IKGEZ6ZQA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.666865058 (728h18m30.579202689s) [noise=true , address=185.167.204.205:4033 , piece_id=EO2VKCTXXE7MA4UOPWABIV23LJDKGMKIUNV3KGFS47OIVXB5WVKQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.670168473 (728h18m30.582506104s) [noise=true , address=185.167.204.205:4009 , piece_id=YQ6P3DOBAPZKVCVTTBI2IJH2UMWWLMLWXMIJNRNSADDSHYF2KGAQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:51.907352601 (728h18m34.819690232s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:24:45.775054746 (728h19m28.687392447s) [noise=true , address=89.149.201.147:4011 , piece_id=FZK5PMW5EYIQLVOLCUOIWR2S42I7BLHX6FRV2MFOTPUKYF36GH2A, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection Apr 5 19:24:45.775168405 (728h19m28.687506076s) [noise=true , address=81.171.14.70:4048 , piece_id=MUDLJPWEYZYTMXLJJZ4FFOTGGVMXZVJSHXE5FZPNHE2SIW6LKPLA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 19:24:45.775267757 (728h19m28.687605408s) [noise=true , address=81.171.14.70:4045 , piece_id=3S6DJCM2FVYHVTFA64VBAWELYHHNKC6RZXBLCUUJS7XY3GACRPIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection Apr 5 19:24:45.775492000 (728h19m28.687829631s) [noise=true , address=81.171.14.70:4046 , piece_id=LMHJQTHJ7JYY35M3F7XWYVD274OKDBDKCSPTPVW75OD45I37WREQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 188.166.225.170:50152->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.775746100 (728h19m28.688083781s) [noise=true , address=81.171.14.70:4047 , piece_id=4GRNNNHVJEPJWTCB6GN65XBRL3TOOFNJEGSC3FNZOACHAFUAPOKA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection Apr 5 19:24:45.776021131 (728h19m28.688358762s) [noise=true , address=81.171.14.70:4031 , piece_id=JOJX7BWJCEIZQPIFFUSB7ZTDB6EY53JN4W4QSQGXTALSWTZYPVPA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection Apr 5 19:24:45.776035619 (728h19m28.68837329s) [noise=true , address=89.149.201.147:4002 , piece_id=L5JCOBLFBXEHYW54OA7WH7ZDH75VSJ2HJJDUXT64TGXWF3TNKSRQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc, address:89.149.201.147:4002): protocol: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection Apr 5 19:24:45.776457844 (728h19m28.688795475s) [noise=true , address=81.171.14.70:4041 , piece_id=U7R4MJPROP5OVX3ZNJS2QFUBBBWMH52G36ISP3QICJSUZ7VO5EDQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: read tcp 188.166.225.170:57676->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.776489063 (728h19m28.688826735s) [seg_pos={0 0}, upload=698890]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Apr 5 19:24:45.942762431 (728h19m28.855100052s) [upload=698890]: Done uploading segments... Apr 5 19:24:45.942795364 (728h19m28.855132995s) [upload=698890]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 4.644095893s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [......X.X...................................................] duration: 4.358327422s trace: https://jaeger.dev.storj.io/trace/71e8b0fffa13491b FAIL (2026-04-05 20:03:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== 3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.445047209 (728h18m29.35738484s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.446660112 (728h18m29.358997743s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:47.230997446 (728h18m30.143335087s) [upload=698890]: Next returned nil segment Apr 5 19:23:47.231015220 (728h18m30.143352852s) [upload=698890]: Waiting for error group managing segments... Apr 5 19:23:47.620452706 (728h18m30.532790347s) [noise=true , address=185.167.204.205:4003 , piece_id=POAEFTZYX3ULC346GTXGT35G2C3CNGGKDBDH24MMLG3L5K6XEUQQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620924707 (728h18m30.533262338s) [noise=true , address=185.167.204.205:4006 , piece_id=OH4TLM4XGF4RFSIMJUCOZTRZAJ2VXSD5IZ272D5O4ZTRJV2EKXFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620995212 (728h18m30.533332843s) [noise=true , address=185.167.204.205:4036 , piece_id=VNGVFT6CPVAFI22JM7YP632ZHFHF63TYOS5OHMPN6SIJYI47Z6ZA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.623436224 (728h18m30.535773855s) [noise=true , address=185.167.204.206:4052 , piece_id=MURH7U3OF7FDWXLNFX2D62OLQXZBRHF5IH2B7O2NIDHYXYVMWPZQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.627891562 (728h18m30.540229193s) [noise=true , address=185.167.204.206:4064 , piece_id=R7LLF3HEHD6HVO4YOBXMNVUSXSOHZL4BDX2RSIL2SKGS3YLPJWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.628832688 (728h18m30.541170319s) [noise=true , address=185.167.204.206:4057 , piece_id=FVDAYDB2M3BS75XT6IWQRSP2QUEVFHSUF3EYAYHHQP37Z2OHS5XA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.629152575 (728h18m30.541490206s) [noise=true , address=185.167.204.206:4069 , piece_id=MWVVC2BOZMC4TWXPVELWHD4RZTBSMYR53CJSIPGQ25B7VDBTMQIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641386105 (728h18m30.553723736s) [noise=true , address=185.167.204.206:4043 , piece_id=VZKY2PYLTFYRGF2JGYWNSHRPEAX6GYTKW7ASDQL7ZALAXXXKNYFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641788261 (728h18m30.554125902s) [noise=true , address=185.167.204.205:4024 , piece_id=2N453JCGF4QS3LKGEZ7BDQZ45NRDPUTBZWECWUKOYBPBRLHJUD3Q, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.643005260 (728h18m30.555342891s) [noise=true , address=185.167.204.205:4011 , piece_id=BG66DKR26PWXMAFTAONWEUFPU6SMOOKMIZWNZSGOIRX2IMUJR4FQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.645249922 (728h18m30.557587553s) [noise=true , address=185.167.204.206:4065 , piece_id=EFD2RBNEIUL4XJ3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651359453 (728h18m30.563697084s) [noise=true , address=185.167.204.206:4056 , piece_id=J5L4SVTQZY7WSPSQ2GXOV7MDN4DIYE454N6FT6K7ZGUD7HR2BFHA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651412215 (728h18m30.563749846s) [noise=true , address=185.167.204.205:4034 , piece_id=VNTYO6Q4IMH2T2XEHJWH64F3YUL45BBCNQLZ46WUZAX62V2SMFTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.654801867 (728h18m30.567139498s) [noise=true , address=185.167.204.206:4071 , piece_id=5NSML2UJOWKUN3FUGXXD37NU3MARIKY6TDCLCDXC6S6WVFV7C5RA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.655101795 (728h18m30.567439426s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.658113638 (728h18m30.570451269s) [noise=true , address=185.167.204.206:4055 , piece_id=VIYGDMLV7V3V76NZ3V5IWBICO6CT7YNTP4BN5QHI542HNFVJ3PTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.662924132 (728h18m30.575261763s) [noise=true , address=185.167.204.206:4062 , piece_id=GRRUPYUEBCY2S6JPGJOEXNJ3OZZDZBYVX27AEMW5YV6IKGEZ6ZQA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.666865058 (728h18m30.579202689s) [noise=true , address=185.167.204.205:4033 , piece_id=EO2VKCTXXE7MA4UOPWABIV23LJDKGMKIUNV3KGFS47OIVXB5WVKQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.670168473 (728h18m30.582506104s) [noise=true , address=185.167.204.205:4009 , piece_id=YQ6P3DOBAPZKVCVTTBI2IJH2UMWWLMLWXMIJNRNSADDSHYF2KGAQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:51.907352601 (728h18m34.819690232s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:24:45.775054746 (728h19m28.687392447s) [noise=true , address=89.149.201.147:4011 , piece_id=FZK5PMW5EYIQLVOLCUOIWR2S42I7BLHX6FRV2MFOTPUKYF36GH2A, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection Apr 5 19:24:45.775168405 (728h19m28.687506076s) [noise=true , address=81.171.14.70:4048 , piece_id=MUDLJPWEYZYTMXLJJZ4FFOTGGVMXZVJSHXE5FZPNHE2SIW6LKPLA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 19:24:45.775267757 (728h19m28.687605408s) [noise=true , address=81.171.14.70:4045 , piece_id=3S6DJCM2FVYHVTFA64VBAWELYHHNKC6RZXBLCUUJS7XY3GACRPIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection Apr 5 19:24:45.775492000 (728h19m28.687829631s) [noise=true , address=81.171.14.70:4046 , piece_id=LMHJQTHJ7JYY35M3F7XWYVD274OKDBDKCSPTPVW75OD45I37WREQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 188.166.225.170:50152->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.775746100 (728h19m28.688083781s) [noise=true , address=81.171.14.70:4047 , piece_id=4GRNNNHVJEPJWTCB6GN65XBRL3TOOFNJEGSC3FNZOACHAFUAPOKA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection Apr 5 19:24:45.776021131 (728h19m28.688358762s) [noise=true , address=81.171.14.70:4031 , piece_id=JOJX7BWJCEIZQPIFFUSB7ZTDB6EY53JN4W4QSQGXTALSWTZYPVPA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection Apr 5 19:24:45.776035619 (728h19m28.68837329s) [noise=true , address=89.149.201.147:4002 , piece_id=L5JCOBLFBXEHYW54OA7WH7ZDH75VSJ2HJJDUXT64TGXWF3TNKSRQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc, address:89.149.201.147:4002): protocol: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection Apr 5 19:24:45.776457844 (728h19m28.688795475s) [noise=true , address=81.171.14.70:4041 , piece_id=U7R4MJPROP5OVX3ZNJS2QFUBBBWMH52G36ISP3QICJSUZ7VO5EDQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: read tcp 188.166.225.170:57676->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.776489063 (728h19m28.688826735s) [seg_pos={0 0}, upload=698890]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Apr 5 19:24:45.942762431 (728h19m28.855100052s) [upload=698890]: Done uploading segments... Apr 5 19:24:45.942795364 (728h19m28.855132995s) [upload=698890]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 4.358360697s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.......X.X..................................................] duration: 4.568050249s trace: https://jaeger.dev.storj.io/trace/71e8b0fffa13491b FAIL (2026-04-05 20:02:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== 3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.445047209 (728h18m29.35738484s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.446660112 (728h18m29.358997743s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:47.230997446 (728h18m30.143335087s) [upload=698890]: Next returned nil segment Apr 5 19:23:47.231015220 (728h18m30.143352852s) [upload=698890]: Waiting for error group managing segments... Apr 5 19:23:47.620452706 (728h18m30.532790347s) [noise=true , address=185.167.204.205:4003 , piece_id=POAEFTZYX3ULC346GTXGT35G2C3CNGGKDBDH24MMLG3L5K6XEUQQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620924707 (728h18m30.533262338s) [noise=true , address=185.167.204.205:4006 , piece_id=OH4TLM4XGF4RFSIMJUCOZTRZAJ2VXSD5IZ272D5O4ZTRJV2EKXFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620995212 (728h18m30.533332843s) [noise=true , address=185.167.204.205:4036 , piece_id=VNGVFT6CPVAFI22JM7YP632ZHFHF63TYOS5OHMPN6SIJYI47Z6ZA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.623436224 (728h18m30.535773855s) [noise=true , address=185.167.204.206:4052 , piece_id=MURH7U3OF7FDWXLNFX2D62OLQXZBRHF5IH2B7O2NIDHYXYVMWPZQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.627891562 (728h18m30.540229193s) [noise=true , address=185.167.204.206:4064 , piece_id=R7LLF3HEHD6HVO4YOBXMNVUSXSOHZL4BDX2RSIL2SKGS3YLPJWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.628832688 (728h18m30.541170319s) [noise=true , address=185.167.204.206:4057 , piece_id=FVDAYDB2M3BS75XT6IWQRSP2QUEVFHSUF3EYAYHHQP37Z2OHS5XA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.629152575 (728h18m30.541490206s) [noise=true , address=185.167.204.206:4069 , piece_id=MWVVC2BOZMC4TWXPVELWHD4RZTBSMYR53CJSIPGQ25B7VDBTMQIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641386105 (728h18m30.553723736s) [noise=true , address=185.167.204.206:4043 , piece_id=VZKY2PYLTFYRGF2JGYWNSHRPEAX6GYTKW7ASDQL7ZALAXXXKNYFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641788261 (728h18m30.554125902s) [noise=true , address=185.167.204.205:4024 , piece_id=2N453JCGF4QS3LKGEZ7BDQZ45NRDPUTBZWECWUKOYBPBRLHJUD3Q, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.643005260 (728h18m30.555342891s) [noise=true , address=185.167.204.205:4011 , piece_id=BG66DKR26PWXMAFTAONWEUFPU6SMOOKMIZWNZSGOIRX2IMUJR4FQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.645249922 (728h18m30.557587553s) [noise=true , address=185.167.204.206:4065 , piece_id=EFD2RBNEIUL4XJ3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651359453 (728h18m30.563697084s) [noise=true , address=185.167.204.206:4056 , piece_id=J5L4SVTQZY7WSPSQ2GXOV7MDN4DIYE454N6FT6K7ZGUD7HR2BFHA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651412215 (728h18m30.563749846s) [noise=true , address=185.167.204.205:4034 , piece_id=VNTYO6Q4IMH2T2XEHJWH64F3YUL45BBCNQLZ46WUZAX62V2SMFTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.654801867 (728h18m30.567139498s) [noise=true , address=185.167.204.206:4071 , piece_id=5NSML2UJOWKUN3FUGXXD37NU3MARIKY6TDCLCDXC6S6WVFV7C5RA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.655101795 (728h18m30.567439426s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.658113638 (728h18m30.570451269s) [noise=true , address=185.167.204.206:4055 , piece_id=VIYGDMLV7V3V76NZ3V5IWBICO6CT7YNTP4BN5QHI542HNFVJ3PTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.662924132 (728h18m30.575261763s) [noise=true , address=185.167.204.206:4062 , piece_id=GRRUPYUEBCY2S6JPGJOEXNJ3OZZDZBYVX27AEMW5YV6IKGEZ6ZQA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.666865058 (728h18m30.579202689s) [noise=true , address=185.167.204.205:4033 , piece_id=EO2VKCTXXE7MA4UOPWABIV23LJDKGMKIUNV3KGFS47OIVXB5WVKQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.670168473 (728h18m30.582506104s) [noise=true , address=185.167.204.205:4009 , piece_id=YQ6P3DOBAPZKVCVTTBI2IJH2UMWWLMLWXMIJNRNSADDSHYF2KGAQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:51.907352601 (728h18m34.819690232s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:24:45.775054746 (728h19m28.687392447s) [noise=true , address=89.149.201.147:4011 , piece_id=FZK5PMW5EYIQLVOLCUOIWR2S42I7BLHX6FRV2MFOTPUKYF36GH2A, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection Apr 5 19:24:45.775168405 (728h19m28.687506076s) [noise=true , address=81.171.14.70:4048 , piece_id=MUDLJPWEYZYTMXLJJZ4FFOTGGVMXZVJSHXE5FZPNHE2SIW6LKPLA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 19:24:45.775267757 (728h19m28.687605408s) [noise=true , address=81.171.14.70:4045 , piece_id=3S6DJCM2FVYHVTFA64VBAWELYHHNKC6RZXBLCUUJS7XY3GACRPIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection Apr 5 19:24:45.775492000 (728h19m28.687829631s) [noise=true , address=81.171.14.70:4046 , piece_id=LMHJQTHJ7JYY35M3F7XWYVD274OKDBDKCSPTPVW75OD45I37WREQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 188.166.225.170:50152->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.775746100 (728h19m28.688083781s) [noise=true , address=81.171.14.70:4047 , piece_id=4GRNNNHVJEPJWTCB6GN65XBRL3TOOFNJEGSC3FNZOACHAFUAPOKA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection Apr 5 19:24:45.776021131 (728h19m28.688358762s) [noise=true , address=81.171.14.70:4031 , piece_id=JOJX7BWJCEIZQPIFFUSB7ZTDB6EY53JN4W4QSQGXTALSWTZYPVPA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection Apr 5 19:24:45.776035619 (728h19m28.68837329s) [noise=true , address=89.149.201.147:4002 , piece_id=L5JCOBLFBXEHYW54OA7WH7ZDH75VSJ2HJJDUXT64TGXWF3TNKSRQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc, address:89.149.201.147:4002): protocol: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection Apr 5 19:24:45.776457844 (728h19m28.688795475s) [noise=true , address=81.171.14.70:4041 , piece_id=U7R4MJPROP5OVX3ZNJS2QFUBBBWMH52G36ISP3QICJSUZ7VO5EDQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: read tcp 188.166.225.170:57676->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.776489063 (728h19m28.688826735s) [seg_pos={0 0}, upload=698890]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Apr 5 19:24:45.942762431 (728h19m28.855100052s) [upload=698890]: Done uploading segments... Apr 5 19:24:45.942795364 (728h19m28.855132995s) [upload=698890]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 4.568066461s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [........X.X.................................................] duration: 3.644761734s trace: https://jaeger.dev.storj.io/trace/71e8b0fffa13491b FAIL (2026-04-05 20:01:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== 3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.445047209 (728h18m29.35738484s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.446660112 (728h18m29.358997743s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:47.230997446 (728h18m30.143335087s) [upload=698890]: Next returned nil segment Apr 5 19:23:47.231015220 (728h18m30.143352852s) [upload=698890]: Waiting for error group managing segments... Apr 5 19:23:47.620452706 (728h18m30.532790347s) [noise=true , address=185.167.204.205:4003 , piece_id=POAEFTZYX3ULC346GTXGT35G2C3CNGGKDBDH24MMLG3L5K6XEUQQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620924707 (728h18m30.533262338s) [noise=true , address=185.167.204.205:4006 , piece_id=OH4TLM4XGF4RFSIMJUCOZTRZAJ2VXSD5IZ272D5O4ZTRJV2EKXFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620995212 (728h18m30.533332843s) [noise=true , address=185.167.204.205:4036 , piece_id=VNGVFT6CPVAFI22JM7YP632ZHFHF63TYOS5OHMPN6SIJYI47Z6ZA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.623436224 (728h18m30.535773855s) [noise=true , address=185.167.204.206:4052 , piece_id=MURH7U3OF7FDWXLNFX2D62OLQXZBRHF5IH2B7O2NIDHYXYVMWPZQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.627891562 (728h18m30.540229193s) [noise=true , address=185.167.204.206:4064 , piece_id=R7LLF3HEHD6HVO4YOBXMNVUSXSOHZL4BDX2RSIL2SKGS3YLPJWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.628832688 (728h18m30.541170319s) [noise=true , address=185.167.204.206:4057 , piece_id=FVDAYDB2M3BS75XT6IWQRSP2QUEVFHSUF3EYAYHHQP37Z2OHS5XA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.629152575 (728h18m30.541490206s) [noise=true , address=185.167.204.206:4069 , piece_id=MWVVC2BOZMC4TWXPVELWHD4RZTBSMYR53CJSIPGQ25B7VDBTMQIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641386105 (728h18m30.553723736s) [noise=true , address=185.167.204.206:4043 , piece_id=VZKY2PYLTFYRGF2JGYWNSHRPEAX6GYTKW7ASDQL7ZALAXXXKNYFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641788261 (728h18m30.554125902s) [noise=true , address=185.167.204.205:4024 , piece_id=2N453JCGF4QS3LKGEZ7BDQZ45NRDPUTBZWECWUKOYBPBRLHJUD3Q, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.643005260 (728h18m30.555342891s) [noise=true , address=185.167.204.205:4011 , piece_id=BG66DKR26PWXMAFTAONWEUFPU6SMOOKMIZWNZSGOIRX2IMUJR4FQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.645249922 (728h18m30.557587553s) [noise=true , address=185.167.204.206:4065 , piece_id=EFD2RBNEIUL4XJ3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651359453 (728h18m30.563697084s) [noise=true , address=185.167.204.206:4056 , piece_id=J5L4SVTQZY7WSPSQ2GXOV7MDN4DIYE454N6FT6K7ZGUD7HR2BFHA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651412215 (728h18m30.563749846s) [noise=true , address=185.167.204.205:4034 , piece_id=VNTYO6Q4IMH2T2XEHJWH64F3YUL45BBCNQLZ46WUZAX62V2SMFTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.654801867 (728h18m30.567139498s) [noise=true , address=185.167.204.206:4071 , piece_id=5NSML2UJOWKUN3FUGXXD37NU3MARIKY6TDCLCDXC6S6WVFV7C5RA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.655101795 (728h18m30.567439426s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.658113638 (728h18m30.570451269s) [noise=true , address=185.167.204.206:4055 , piece_id=VIYGDMLV7V3V76NZ3V5IWBICO6CT7YNTP4BN5QHI542HNFVJ3PTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.662924132 (728h18m30.575261763s) [noise=true , address=185.167.204.206:4062 , piece_id=GRRUPYUEBCY2S6JPGJOEXNJ3OZZDZBYVX27AEMW5YV6IKGEZ6ZQA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.666865058 (728h18m30.579202689s) [noise=true , address=185.167.204.205:4033 , piece_id=EO2VKCTXXE7MA4UOPWABIV23LJDKGMKIUNV3KGFS47OIVXB5WVKQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.670168473 (728h18m30.582506104s) [noise=true , address=185.167.204.205:4009 , piece_id=YQ6P3DOBAPZKVCVTTBI2IJH2UMWWLMLWXMIJNRNSADDSHYF2KGAQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:51.907352601 (728h18m34.819690232s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:24:45.775054746 (728h19m28.687392447s) [noise=true , address=89.149.201.147:4011 , piece_id=FZK5PMW5EYIQLVOLCUOIWR2S42I7BLHX6FRV2MFOTPUKYF36GH2A, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection Apr 5 19:24:45.775168405 (728h19m28.687506076s) [noise=true , address=81.171.14.70:4048 , piece_id=MUDLJPWEYZYTMXLJJZ4FFOTGGVMXZVJSHXE5FZPNHE2SIW6LKPLA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 19:24:45.775267757 (728h19m28.687605408s) [noise=true , address=81.171.14.70:4045 , piece_id=3S6DJCM2FVYHVTFA64VBAWELYHHNKC6RZXBLCUUJS7XY3GACRPIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection Apr 5 19:24:45.775492000 (728h19m28.687829631s) [noise=true , address=81.171.14.70:4046 , piece_id=LMHJQTHJ7JYY35M3F7XWYVD274OKDBDKCSPTPVW75OD45I37WREQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 188.166.225.170:50152->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.775746100 (728h19m28.688083781s) [noise=true , address=81.171.14.70:4047 , piece_id=4GRNNNHVJEPJWTCB6GN65XBRL3TOOFNJEGSC3FNZOACHAFUAPOKA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection Apr 5 19:24:45.776021131 (728h19m28.688358762s) [noise=true , address=81.171.14.70:4031 , piece_id=JOJX7BWJCEIZQPIFFUSB7ZTDB6EY53JN4W4QSQGXTALSWTZYPVPA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection Apr 5 19:24:45.776035619 (728h19m28.68837329s) [noise=true , address=89.149.201.147:4002 , piece_id=L5JCOBLFBXEHYW54OA7WH7ZDH75VSJ2HJJDUXT64TGXWF3TNKSRQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc, address:89.149.201.147:4002): protocol: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection Apr 5 19:24:45.776457844 (728h19m28.688795475s) [noise=true , address=81.171.14.70:4041 , piece_id=U7R4MJPROP5OVX3ZNJS2QFUBBBWMH52G36ISP3QICJSUZ7VO5EDQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: read tcp 188.166.225.170:57676->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.776489063 (728h19m28.688826735s) [seg_pos={0 0}, upload=698890]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Apr 5 19:24:45.942762431 (728h19m28.855100052s) [upload=698890]: Done uploading segments... Apr 5 19:24:45.942795364 (728h19m28.855132995s) [upload=698890]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 3.644789849s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.........X.X................................................] duration: 4.509810225s trace: https://jaeger.dev.storj.io/trace/71e8b0fffa13491b FAIL (2026-04-05 20:00:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== 3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.445047209 (728h18m29.35738484s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:46.446660112 (728h18m29.358997743s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Uploading piece... Apr 5 19:23:47.230997446 (728h18m30.143335087s) [upload=698890]: Next returned nil segment Apr 5 19:23:47.231015220 (728h18m30.143352852s) [upload=698890]: Waiting for error group managing segments... Apr 5 19:23:47.620452706 (728h18m30.532790347s) [noise=true , address=185.167.204.205:4003 , piece_id=POAEFTZYX3ULC346GTXGT35G2C3CNGGKDBDH24MMLG3L5K6XEUQQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620924707 (728h18m30.533262338s) [noise=true , address=185.167.204.205:4006 , piece_id=OH4TLM4XGF4RFSIMJUCOZTRZAJ2VXSD5IZ272D5O4ZTRJV2EKXFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.620995212 (728h18m30.533332843s) [noise=true , address=185.167.204.205:4036 , piece_id=VNGVFT6CPVAFI22JM7YP632ZHFHF63TYOS5OHMPN6SIJYI47Z6ZA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.623436224 (728h18m30.535773855s) [noise=true , address=185.167.204.206:4052 , piece_id=MURH7U3OF7FDWXLNFX2D62OLQXZBRHF5IH2B7O2NIDHYXYVMWPZQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.627891562 (728h18m30.540229193s) [noise=true , address=185.167.204.206:4064 , piece_id=R7LLF3HEHD6HVO4YOBXMNVUSXSOHZL4BDX2RSIL2SKGS3YLPJWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.628832688 (728h18m30.541170319s) [noise=true , address=185.167.204.206:4057 , piece_id=FVDAYDB2M3BS75XT6IWQRSP2QUEVFHSUF3EYAYHHQP37Z2OHS5XA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.629152575 (728h18m30.541490206s) [noise=true , address=185.167.204.206:4069 , piece_id=MWVVC2BOZMC4TWXPVELWHD4RZTBSMYR53CJSIPGQ25B7VDBTMQIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641386105 (728h18m30.553723736s) [noise=true , address=185.167.204.206:4043 , piece_id=VZKY2PYLTFYRGF2JGYWNSHRPEAX6GYTKW7ASDQL7ZALAXXXKNYFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.641788261 (728h18m30.554125902s) [noise=true , address=185.167.204.205:4024 , piece_id=2N453JCGF4QS3LKGEZ7BDQZ45NRDPUTBZWECWUKOYBPBRLHJUD3Q, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.643005260 (728h18m30.555342891s) [noise=true , address=185.167.204.205:4011 , piece_id=BG66DKR26PWXMAFTAONWEUFPU6SMOOKMIZWNZSGOIRX2IMUJR4FQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.645249922 (728h18m30.557587553s) [noise=true , address=185.167.204.206:4065 , piece_id=EFD2RBNEIUL4XJ3J4R74552BXEVHR22INNGBRRWNEB7YJFCXKFFA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651359453 (728h18m30.563697084s) [noise=true , address=185.167.204.206:4056 , piece_id=J5L4SVTQZY7WSPSQ2GXOV7MDN4DIYE454N6FT6K7ZGUD7HR2BFHA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.651412215 (728h18m30.563749846s) [noise=true , address=185.167.204.205:4034 , piece_id=VNTYO6Q4IMH2T2XEHJWH64F3YUL45BBCNQLZ46WUZAX62V2SMFTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.654801867 (728h18m30.567139498s) [noise=true , address=185.167.204.206:4071 , piece_id=5NSML2UJOWKUN3FUGXXD37NU3MARIKY6TDCLCDXC6S6WVFV7C5RA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.655101795 (728h18m30.567439426s) [noise=true , address=185.167.204.205:4014 , piece_id=N3MPWVZ2RTISLPAJFFSURQNRNDWY6KMBFZOONAX3U74GLSFZKWDA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.658113638 (728h18m30.570451269s) [noise=true , address=185.167.204.206:4055 , piece_id=VIYGDMLV7V3V76NZ3V5IWBICO6CT7YNTP4BN5QHI542HNFVJ3PTA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.662924132 (728h18m30.575261763s) [noise=true , address=185.167.204.206:4062 , piece_id=GRRUPYUEBCY2S6JPGJOEXNJ3OZZDZBYVX27AEMW5YV6IKGEZ6ZQA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.666865058 (728h18m30.579202689s) [noise=true , address=185.167.204.205:4033 , piece_id=EO2VKCTXXE7MA4UOPWABIV23LJDKGMKIUNV3KGFS47OIVXB5WVKQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:47.670168473 (728h18m30.582506104s) [noise=true , address=185.167.204.205:4009 , piece_id=YQ6P3DOBAPZKVCVTTBI2IJH2UMWWLMLWXMIJNRNSADDSHYF2KGAQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:23:51.907352601 (728h18m34.819690232s) [noise=true , address=89.149.201.147:4003 , piece_id=EDENB2NCQRRUJEKQGC7DMMX6ZF4IIWV6WUV7YTW7W55BJQU7QNWQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: <nil> Apr 5 19:24:45.775054746 (728h19m28.687392447s) [noise=true , address=89.149.201.147:4011 , piece_id=FZK5PMW5EYIQLVOLCUOIWR2S42I7BLHX6FRV2MFOTPUKYF36GH2A, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48780->89.149.201.147:4011: use of closed network connection Apr 5 19:24:45.775168405 (728h19m28.687506076s) [noise=true , address=81.171.14.70:4048 , piece_id=MUDLJPWEYZYTMXLJJZ4FFOTGGVMXZVJSHXE5FZPNHE2SIW6LKPLA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 19:24:45.775267757 (728h19m28.687605408s) [noise=true , address=81.171.14.70:4045 , piece_id=3S6DJCM2FVYHVTFA64VBAWELYHHNKC6RZXBLCUUJS7XY3GACRPIQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33260->81.171.14.70:4045: use of closed network connection Apr 5 19:24:45.775492000 (728h19m28.687829631s) [noise=true , address=81.171.14.70:4046 , piece_id=LMHJQTHJ7JYY35M3F7XWYVD274OKDBDKCSPTPVW75OD45I37WREQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 188.166.225.170:50152->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.775746100 (728h19m28.688083781s) [noise=true , address=81.171.14.70:4047 , piece_id=4GRNNNHVJEPJWTCB6GN65XBRL3TOOFNJEGSC3FNZOACHAFUAPOKA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56156->81.171.14.70:4047: use of closed network connection Apr 5 19:24:45.776021131 (728h19m28.688358762s) [noise=true , address=81.171.14.70:4031 , piece_id=JOJX7BWJCEIZQPIFFUSB7ZTDB6EY53JN4W4QSQGXTALSWTZYPVPA, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36640->81.171.14.70:4031: use of closed network connection Apr 5 19:24:45.776035619 (728h19m28.68837329s) [noise=true , address=89.149.201.147:4002 , piece_id=L5JCOBLFBXEHYW54OA7WH7ZDH75VSJ2HJJDUXT64TGXWF3TNKSRQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12RDCCUcFecPi2tRuk69SueE84EdSssocDPB4WEmUQaXGacDAhc, address:89.149.201.147:4002): protocol: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33962->89.149.201.147:4002: use of closed network connection Apr 5 19:24:45.776457844 (728h19m28.688795475s) [noise=true , address=81.171.14.70:4041 , piece_id=U7R4MJPROP5OVX3ZNJS2QFUBBBWMH52G36ISP3QICJSUZ7VO5EDQ, seg_pos={0 0}, upload=698890]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: read tcp 188.166.225.170:57676->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 19:24:45.776489063 (728h19m28.688826735s) [seg_pos={0 0}, upload=698890]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Apr 5 19:24:45.942762431 (728h19m28.855100052s) [upload=698890]: Done uploading segments... Apr 5 19:24:45.942795364 (728h19m28.855132995s) [upload=698890]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 4.509831746s core uplink eu-select-1 eu1-placement-15

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)