Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.....X....X..X........XX.XX..X...............X.......X......] duration: 15.757803396s FAIL (2025-09-07 15:40:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:36:03.192216018 (26h7m39.231789138s) [upload=23004]: Starting upload Sep 7 15:36:03.192274741 (26h7m39.231847861s) [upload=23004]: Done starting upload Sep 7 15:36:03.192338963 (26h7m39.231912083s) [upload=23004]: Uploading segments... Sep 7 15:36:03.192454144 (26h7m39.232027264s) [upload=23004]: Got next segment. Inline: false Sep 7 15:36:03.192482228 (26h7m39.232055338s) [upload=23004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:36:03.192486767 (26h7m39.232059887s) [upload=23004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:36:03.487935637 (26h7m39.527508757s) [seg_pos={0 0}, upload=23004]: Begin upload segment... Sep 7 15:36:03.487980703 (26h7m39.527553823s) [seg_pos={0 0}, upload=23004]: Done begin upload segment. Sep 7 15:36:03.487999539 (26h7m39.527572659s) [noise=true , address=77.90.31.19:4842 , piece_id=3MVZTN47VLDKNXVRTGLAZ3MDLUT3GT4VKM7KMWLWTNJDKIDUHYHA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488113476 (26h7m39.527686596s) [noise=true , address=77.90.31.12:4590 , piece_id=2244PECVOTKU26LAELDDEOFMKAOCJIOPWBEK3IGBE5SB5E2VMMJA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488477183 (26h7m39.528050303s) [noise=true , address=45.91.116.4:4939 , piece_id=QMWLG3GX6PY5IA3PVUDWXHXAA6SRORXKASIHUPDJGBKNQQHEHIPQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488635737 (26h7m39.528208857s) [noise=true , address=45.91.116.7:4011 , piece_id=G7ZYPHTSX4R2EFI5ZZFN7ACVF6E53XT5OLQG2I6VVJEFYXQ3IIKQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488812204 (26h7m39.528385324s) [noise=true , address=77.90.31.10:4518 , piece_id=THTBURY3HOWHHEGR3S6ZRPECXNAO3M5D26ATLHJHZR4SWXWAMY2Q, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488958824 (26h7m39.528531944s) [noise=true , address=45.91.116.9:4261 , piece_id=QDOA5NPWGPWPDH6XZYJD7UAF3TTZHCJI47DVIAPFE6HF3CCF67EQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489097559 (26h7m39.5286707s) [noise=true , address=77.90.31.19:4841 , piece_id=UE2V2DI7EYDJWBPGIEHT2TI22HB22LPITW5FTEXIUJEOTU67GXMQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489302172 (26h7m39.528875292s) [noise=true , address=77.90.31.13:4628 , piece_id=OZP6LRGQ6WCDLHDVQISDVUEFRZAEUJ3UDCZ36HBLR25O6XBTB2NA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489435627 (26h7m39.529008747s) [noise=true , address=45.91.116.8:4048 , piece_id=3I5YOPQ7PTGWH5PAL74QSMLHVKPPX6BWTWTH47YM6QJU33XCEBJQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489574192 (26h7m39.529147312s) [noise=true , address=77.90.31.19:4843 , piece_id=XFDYVPVOBBQCIQMPR4JCLLEKPVGEAM4YJPOSJIHPLC6VDPUROCIA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489661028 (26h7m39.529234148s) [noise=true , address=77.90.31.16:4734 , piece_id=SP7SAVF572CM5CTE6VYS67HQCAKZQ34J4FRZEWGGNX6CIRKRG53A, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489896860 (26h7m39.52946998s) [noise=true , address=45.91.116.13:4154 , piece_id=YFYTQG3ZEYWRBBEMZ2PF4UIUK7VD2YBDP4GWVZL7XS754ZUQQ7SA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490009024 (26h7m39.529582144s) [noise=true , address=45.91.116.14:4189 , piece_id=MHAVVRDB3DC6VZ6NZQKRXEYWGKYWBQXLEGBS73TVEEZASOTT5XRA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490112943 (26h7m39.529686063s) [noise=true , address=77.90.31.14:4663 , piece_id=FEL4QHHHUXPGFYWBFTFMO346E5MEJHYSJIEQEQA6T5D6EUK5GSFA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490277498 (26h7m39.529850618s) [noise=true , address=77.90.31.15:4697 , piece_id=4X2HZTVC24PARUSP7ZFJQQBP644BKJY7ZS6QPONEDFCNKROMENVQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490575549 (26h7m39.530148659s) [noise=true , address=45.91.116.12:4226 , piece_id=S6ULWGIZZHY56GQ7LQ47H25NVEBPS3UHU5QYPLUT4BEBZ2GXZXMA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:05.499052531 (26h7m41.538625651s) [upload=23004]: Next returned nil segment Sep 7 15:36:05.499070735 (26h7m41.538643845s) [upload=23004]: Waiting for error group managing segments... Sep 7 15:36:05.885906268 (26h7m41.925479388s) [noise=true , address=45.91.116.9:4261 , piece_id=QDOA5NPWGPWPDH6XZYJD7UAF3TTZHCJI47DVIAPFE6HF3CCF67EQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.897076963 (26h7m41.936650083s) [noise=true , address=45.91.116.4:4939 , piece_id=QMWLG3GX6PY5IA3PVUDWXHXAA6SRORXKASIHUPDJGBKNQQHEHIPQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.923811300 (26h7m41.96338442s) [noise=true , address=45.91.116.13:4154 , piece_id=YFYTQG3ZEYWRBBEMZ2PF4UIUK7VD2YBDP4GWVZL7XS754ZUQQ7SA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.942724120 (26h7m41.98229723s) [noise=true , address=77.90.31.19:4842 , piece_id=3MVZTN47VLDKNXVRTGLAZ3MDLUT3GT4VKM7KMWLWTNJDKIDUHYHA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.953070108 (26h7m41.992643228s) [noise=true , address=77.90.31.12:4590 , piece_id=2244PECVOTKU26LAELDDEOFMKAOCJIOPWBEK3IGBE5SB5E2VMMJA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.961410377 (26h7m42.000983497s) [noise=true , address=77.90.31.15:4697 , piece_id=4X2HZTVC24PARUSP7ZFJQQBP644BKJY7ZS6QPONEDFCNKROMENVQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.997943784 (26h7m42.037516894s) [noise=true , address=45.91.116.14:4189 , piece_id=MHAVVRDB3DC6VZ6NZQKRXEYWGKYWBQXLEGBS73TVEEZASOTT5XRA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.003770574 (26h7m42.043343694s) [noise=true , address=77.90.31.14:4663 , piece_id=FEL4QHHHUXPGFYWBFTFMO346E5MEJHYSJIEQEQA6T5D6EUK5GSFA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.071696002 (26h7m42.111269122s) [noise=true , address=77.90.31.10:4518 , piece_id=THTBURY3HOWHHEGR3S6ZRPECXNAO3M5D26ATLHJHZR4SWXWAMY2Q, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.107506336 (26h7m42.147079446s) [noise=true , address=77.90.31.19:4843 , piece_id=XFDYVPVOBBQCIQMPR4JCLLEKPVGEAM4YJPOSJIHPLC6VDPUROCIA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.128114600 (26h7m42.16768772s) [noise=true , address=45.91.116.8:4048 , piece_id=3I5YOPQ7PTGWH5PAL74QSMLHVKPPX6BWTWTH47YM6QJU33XCEBJQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.265087974 (26h7m42.304661094s) [noise=true , address=77.90.31.16:4734 , piece_id=SP7SAVF572CM5CTE6VYS67HQCAKZQ34J4FRZEWGGNX6CIRKRG53A, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.389194034 (26h7m42.428767144s) [noise=true , address=77.90.31.19:4841 , piece_id=UE2V2DI7EYDJWBPGIEHT2TI22HB22LPITW5FTEXIUJEOTU67GXMQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.389221016 (26h7m42.428794136s) [seg_pos={0 0}, upload=23004]: Segment reached optimal threshold of 13 pieces. Sep 7 15:36:06.389324835 (26h7m42.428897955s) [noise=true , address=45.91.116.7:4011 , piece_id=G7ZYPHTSX4R2EFI5ZZFN7ACVF6E53XT5OLQG2I6VVJEFYXQ3IIKQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Sep 7 15:36:06.389478469 (26h7m42.429051589s) [noise=true , address=77.90.31.13:4628 , piece_id=OZP6LRGQ6WCDLHDVQISDVUEFRZAEUJ3UDCZ36HBLR25O6XBTB2NA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection; write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection Sep 7 15:36:06.389496152 (26h7m42.429069272s) [noise=true , address=45.91.116.12:4226 , piece_id=S6ULWGIZZHY56GQ7LQ47H25NVEBPS3UHU5QYPLUT4BEBZ2GXZXMA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection; write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection Sep 7 15:36:06.389538985 (26h7m42.429112105s) [seg_pos={0 0}, upload=23004]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:36:06.389608707 (26h7m42.429181827s) [upload=23004]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:36:06.389612845 (26h7m42.429185965s) [upload=23004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:36:06.739578335 (26h7m42.779151455s) [upload=23004]: Done uploading segments... Sep 7 15:36:06.739598183 (26h7m42.779171294s) [upload=23004]: Upload finished. err: <nil> Duration: 15.757830677s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [......X....X..X........XX.XX..X...............X.......X.....] duration: 15.117099276s FAIL (2025-09-07 15:39:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:36:03.192216018 (26h7m39.231789138s) [upload=23004]: Starting upload Sep 7 15:36:03.192274741 (26h7m39.231847861s) [upload=23004]: Done starting upload Sep 7 15:36:03.192338963 (26h7m39.231912083s) [upload=23004]: Uploading segments... Sep 7 15:36:03.192454144 (26h7m39.232027264s) [upload=23004]: Got next segment. Inline: false Sep 7 15:36:03.192482228 (26h7m39.232055338s) [upload=23004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:36:03.192486767 (26h7m39.232059887s) [upload=23004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:36:03.487935637 (26h7m39.527508757s) [seg_pos={0 0}, upload=23004]: Begin upload segment... Sep 7 15:36:03.487980703 (26h7m39.527553823s) [seg_pos={0 0}, upload=23004]: Done begin upload segment. Sep 7 15:36:03.487999539 (26h7m39.527572659s) [noise=true , address=77.90.31.19:4842 , piece_id=3MVZTN47VLDKNXVRTGLAZ3MDLUT3GT4VKM7KMWLWTNJDKIDUHYHA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488113476 (26h7m39.527686596s) [noise=true , address=77.90.31.12:4590 , piece_id=2244PECVOTKU26LAELDDEOFMKAOCJIOPWBEK3IGBE5SB5E2VMMJA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488477183 (26h7m39.528050303s) [noise=true , address=45.91.116.4:4939 , piece_id=QMWLG3GX6PY5IA3PVUDWXHXAA6SRORXKASIHUPDJGBKNQQHEHIPQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488635737 (26h7m39.528208857s) [noise=true , address=45.91.116.7:4011 , piece_id=G7ZYPHTSX4R2EFI5ZZFN7ACVF6E53XT5OLQG2I6VVJEFYXQ3IIKQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488812204 (26h7m39.528385324s) [noise=true , address=77.90.31.10:4518 , piece_id=THTBURY3HOWHHEGR3S6ZRPECXNAO3M5D26ATLHJHZR4SWXWAMY2Q, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488958824 (26h7m39.528531944s) [noise=true , address=45.91.116.9:4261 , piece_id=QDOA5NPWGPWPDH6XZYJD7UAF3TTZHCJI47DVIAPFE6HF3CCF67EQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489097559 (26h7m39.5286707s) [noise=true , address=77.90.31.19:4841 , piece_id=UE2V2DI7EYDJWBPGIEHT2TI22HB22LPITW5FTEXIUJEOTU67GXMQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489302172 (26h7m39.528875292s) [noise=true , address=77.90.31.13:4628 , piece_id=OZP6LRGQ6WCDLHDVQISDVUEFRZAEUJ3UDCZ36HBLR25O6XBTB2NA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489435627 (26h7m39.529008747s) [noise=true , address=45.91.116.8:4048 , piece_id=3I5YOPQ7PTGWH5PAL74QSMLHVKPPX6BWTWTH47YM6QJU33XCEBJQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489574192 (26h7m39.529147312s) [noise=true , address=77.90.31.19:4843 , piece_id=XFDYVPVOBBQCIQMPR4JCLLEKPVGEAM4YJPOSJIHPLC6VDPUROCIA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489661028 (26h7m39.529234148s) [noise=true , address=77.90.31.16:4734 , piece_id=SP7SAVF572CM5CTE6VYS67HQCAKZQ34J4FRZEWGGNX6CIRKRG53A, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489896860 (26h7m39.52946998s) [noise=true , address=45.91.116.13:4154 , piece_id=YFYTQG3ZEYWRBBEMZ2PF4UIUK7VD2YBDP4GWVZL7XS754ZUQQ7SA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490009024 (26h7m39.529582144s) [noise=true , address=45.91.116.14:4189 , piece_id=MHAVVRDB3DC6VZ6NZQKRXEYWGKYWBQXLEGBS73TVEEZASOTT5XRA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490112943 (26h7m39.529686063s) [noise=true , address=77.90.31.14:4663 , piece_id=FEL4QHHHUXPGFYWBFTFMO346E5MEJHYSJIEQEQA6T5D6EUK5GSFA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490277498 (26h7m39.529850618s) [noise=true , address=77.90.31.15:4697 , piece_id=4X2HZTVC24PARUSP7ZFJQQBP644BKJY7ZS6QPONEDFCNKROMENVQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490575549 (26h7m39.530148659s) [noise=true , address=45.91.116.12:4226 , piece_id=S6ULWGIZZHY56GQ7LQ47H25NVEBPS3UHU5QYPLUT4BEBZ2GXZXMA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:05.499052531 (26h7m41.538625651s) [upload=23004]: Next returned nil segment Sep 7 15:36:05.499070735 (26h7m41.538643845s) [upload=23004]: Waiting for error group managing segments... Sep 7 15:36:05.885906268 (26h7m41.925479388s) [noise=true , address=45.91.116.9:4261 , piece_id=QDOA5NPWGPWPDH6XZYJD7UAF3TTZHCJI47DVIAPFE6HF3CCF67EQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.897076963 (26h7m41.936650083s) [noise=true , address=45.91.116.4:4939 , piece_id=QMWLG3GX6PY5IA3PVUDWXHXAA6SRORXKASIHUPDJGBKNQQHEHIPQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.923811300 (26h7m41.96338442s) [noise=true , address=45.91.116.13:4154 , piece_id=YFYTQG3ZEYWRBBEMZ2PF4UIUK7VD2YBDP4GWVZL7XS754ZUQQ7SA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.942724120 (26h7m41.98229723s) [noise=true , address=77.90.31.19:4842 , piece_id=3MVZTN47VLDKNXVRTGLAZ3MDLUT3GT4VKM7KMWLWTNJDKIDUHYHA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.953070108 (26h7m41.992643228s) [noise=true , address=77.90.31.12:4590 , piece_id=2244PECVOTKU26LAELDDEOFMKAOCJIOPWBEK3IGBE5SB5E2VMMJA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.961410377 (26h7m42.000983497s) [noise=true , address=77.90.31.15:4697 , piece_id=4X2HZTVC24PARUSP7ZFJQQBP644BKJY7ZS6QPONEDFCNKROMENVQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.997943784 (26h7m42.037516894s) [noise=true , address=45.91.116.14:4189 , piece_id=MHAVVRDB3DC6VZ6NZQKRXEYWGKYWBQXLEGBS73TVEEZASOTT5XRA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.003770574 (26h7m42.043343694s) [noise=true , address=77.90.31.14:4663 , piece_id=FEL4QHHHUXPGFYWBFTFMO346E5MEJHYSJIEQEQA6T5D6EUK5GSFA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.071696002 (26h7m42.111269122s) [noise=true , address=77.90.31.10:4518 , piece_id=THTBURY3HOWHHEGR3S6ZRPECXNAO3M5D26ATLHJHZR4SWXWAMY2Q, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.107506336 (26h7m42.147079446s) [noise=true , address=77.90.31.19:4843 , piece_id=XFDYVPVOBBQCIQMPR4JCLLEKPVGEAM4YJPOSJIHPLC6VDPUROCIA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.128114600 (26h7m42.16768772s) [noise=true , address=45.91.116.8:4048 , piece_id=3I5YOPQ7PTGWH5PAL74QSMLHVKPPX6BWTWTH47YM6QJU33XCEBJQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.265087974 (26h7m42.304661094s) [noise=true , address=77.90.31.16:4734 , piece_id=SP7SAVF572CM5CTE6VYS67HQCAKZQ34J4FRZEWGGNX6CIRKRG53A, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.389194034 (26h7m42.428767144s) [noise=true , address=77.90.31.19:4841 , piece_id=UE2V2DI7EYDJWBPGIEHT2TI22HB22LPITW5FTEXIUJEOTU67GXMQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.389221016 (26h7m42.428794136s) [seg_pos={0 0}, upload=23004]: Segment reached optimal threshold of 13 pieces. Sep 7 15:36:06.389324835 (26h7m42.428897955s) [noise=true , address=45.91.116.7:4011 , piece_id=G7ZYPHTSX4R2EFI5ZZFN7ACVF6E53XT5OLQG2I6VVJEFYXQ3IIKQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Sep 7 15:36:06.389478469 (26h7m42.429051589s) [noise=true , address=77.90.31.13:4628 , piece_id=OZP6LRGQ6WCDLHDVQISDVUEFRZAEUJ3UDCZ36HBLR25O6XBTB2NA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection; write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection Sep 7 15:36:06.389496152 (26h7m42.429069272s) [noise=true , address=45.91.116.12:4226 , piece_id=S6ULWGIZZHY56GQ7LQ47H25NVEBPS3UHU5QYPLUT4BEBZ2GXZXMA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection; write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection Sep 7 15:36:06.389538985 (26h7m42.429112105s) [seg_pos={0 0}, upload=23004]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:36:06.389608707 (26h7m42.429181827s) [upload=23004]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:36:06.389612845 (26h7m42.429185965s) [upload=23004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:36:06.739578335 (26h7m42.779151455s) [upload=23004]: Done uploading segments... Sep 7 15:36:06.739598183 (26h7m42.779171294s) [upload=23004]: Upload finished. err: <nil> Duration: 15.117124796s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [X......X....X..X........XX.XX..X...............X.......X....] duration: 7.184876623s FAIL (2025-09-07 15:39:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:36:03.192216018 (26h7m39.231789138s) [upload=23004]: Starting upload Sep 7 15:36:03.192274741 (26h7m39.231847861s) [upload=23004]: Done starting upload Sep 7 15:36:03.192338963 (26h7m39.231912083s) [upload=23004]: Uploading segments... Sep 7 15:36:03.192454144 (26h7m39.232027264s) [upload=23004]: Got next segment. Inline: false Sep 7 15:36:03.192482228 (26h7m39.232055338s) [upload=23004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:36:03.192486767 (26h7m39.232059887s) [upload=23004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:36:03.487935637 (26h7m39.527508757s) [seg_pos={0 0}, upload=23004]: Begin upload segment... Sep 7 15:36:03.487980703 (26h7m39.527553823s) [seg_pos={0 0}, upload=23004]: Done begin upload segment. Sep 7 15:36:03.487999539 (26h7m39.527572659s) [noise=true , address=77.90.31.19:4842 , piece_id=3MVZTN47VLDKNXVRTGLAZ3MDLUT3GT4VKM7KMWLWTNJDKIDUHYHA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488113476 (26h7m39.527686596s) [noise=true , address=77.90.31.12:4590 , piece_id=2244PECVOTKU26LAELDDEOFMKAOCJIOPWBEK3IGBE5SB5E2VMMJA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488477183 (26h7m39.528050303s) [noise=true , address=45.91.116.4:4939 , piece_id=QMWLG3GX6PY5IA3PVUDWXHXAA6SRORXKASIHUPDJGBKNQQHEHIPQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488635737 (26h7m39.528208857s) [noise=true , address=45.91.116.7:4011 , piece_id=G7ZYPHTSX4R2EFI5ZZFN7ACVF6E53XT5OLQG2I6VVJEFYXQ3IIKQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488812204 (26h7m39.528385324s) [noise=true , address=77.90.31.10:4518 , piece_id=THTBURY3HOWHHEGR3S6ZRPECXNAO3M5D26ATLHJHZR4SWXWAMY2Q, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488958824 (26h7m39.528531944s) [noise=true , address=45.91.116.9:4261 , piece_id=QDOA5NPWGPWPDH6XZYJD7UAF3TTZHCJI47DVIAPFE6HF3CCF67EQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489097559 (26h7m39.5286707s) [noise=true , address=77.90.31.19:4841 , piece_id=UE2V2DI7EYDJWBPGIEHT2TI22HB22LPITW5FTEXIUJEOTU67GXMQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489302172 (26h7m39.528875292s) [noise=true , address=77.90.31.13:4628 , piece_id=OZP6LRGQ6WCDLHDVQISDVUEFRZAEUJ3UDCZ36HBLR25O6XBTB2NA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489435627 (26h7m39.529008747s) [noise=true , address=45.91.116.8:4048 , piece_id=3I5YOPQ7PTGWH5PAL74QSMLHVKPPX6BWTWTH47YM6QJU33XCEBJQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489574192 (26h7m39.529147312s) [noise=true , address=77.90.31.19:4843 , piece_id=XFDYVPVOBBQCIQMPR4JCLLEKPVGEAM4YJPOSJIHPLC6VDPUROCIA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489661028 (26h7m39.529234148s) [noise=true , address=77.90.31.16:4734 , piece_id=SP7SAVF572CM5CTE6VYS67HQCAKZQ34J4FRZEWGGNX6CIRKRG53A, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489896860 (26h7m39.52946998s) [noise=true , address=45.91.116.13:4154 , piece_id=YFYTQG3ZEYWRBBEMZ2PF4UIUK7VD2YBDP4GWVZL7XS754ZUQQ7SA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490009024 (26h7m39.529582144s) [noise=true , address=45.91.116.14:4189 , piece_id=MHAVVRDB3DC6VZ6NZQKRXEYWGKYWBQXLEGBS73TVEEZASOTT5XRA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490112943 (26h7m39.529686063s) [noise=true , address=77.90.31.14:4663 , piece_id=FEL4QHHHUXPGFYWBFTFMO346E5MEJHYSJIEQEQA6T5D6EUK5GSFA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490277498 (26h7m39.529850618s) [noise=true , address=77.90.31.15:4697 , piece_id=4X2HZTVC24PARUSP7ZFJQQBP644BKJY7ZS6QPONEDFCNKROMENVQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490575549 (26h7m39.530148659s) [noise=true , address=45.91.116.12:4226 , piece_id=S6ULWGIZZHY56GQ7LQ47H25NVEBPS3UHU5QYPLUT4BEBZ2GXZXMA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:05.499052531 (26h7m41.538625651s) [upload=23004]: Next returned nil segment Sep 7 15:36:05.499070735 (26h7m41.538643845s) [upload=23004]: Waiting for error group managing segments... Sep 7 15:36:05.885906268 (26h7m41.925479388s) [noise=true , address=45.91.116.9:4261 , piece_id=QDOA5NPWGPWPDH6XZYJD7UAF3TTZHCJI47DVIAPFE6HF3CCF67EQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.897076963 (26h7m41.936650083s) [noise=true , address=45.91.116.4:4939 , piece_id=QMWLG3GX6PY5IA3PVUDWXHXAA6SRORXKASIHUPDJGBKNQQHEHIPQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.923811300 (26h7m41.96338442s) [noise=true , address=45.91.116.13:4154 , piece_id=YFYTQG3ZEYWRBBEMZ2PF4UIUK7VD2YBDP4GWVZL7XS754ZUQQ7SA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.942724120 (26h7m41.98229723s) [noise=true , address=77.90.31.19:4842 , piece_id=3MVZTN47VLDKNXVRTGLAZ3MDLUT3GT4VKM7KMWLWTNJDKIDUHYHA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.953070108 (26h7m41.992643228s) [noise=true , address=77.90.31.12:4590 , piece_id=2244PECVOTKU26LAELDDEOFMKAOCJIOPWBEK3IGBE5SB5E2VMMJA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.961410377 (26h7m42.000983497s) [noise=true , address=77.90.31.15:4697 , piece_id=4X2HZTVC24PARUSP7ZFJQQBP644BKJY7ZS6QPONEDFCNKROMENVQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.997943784 (26h7m42.037516894s) [noise=true , address=45.91.116.14:4189 , piece_id=MHAVVRDB3DC6VZ6NZQKRXEYWGKYWBQXLEGBS73TVEEZASOTT5XRA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.003770574 (26h7m42.043343694s) [noise=true , address=77.90.31.14:4663 , piece_id=FEL4QHHHUXPGFYWBFTFMO346E5MEJHYSJIEQEQA6T5D6EUK5GSFA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.071696002 (26h7m42.111269122s) [noise=true , address=77.90.31.10:4518 , piece_id=THTBURY3HOWHHEGR3S6ZRPECXNAO3M5D26ATLHJHZR4SWXWAMY2Q, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.107506336 (26h7m42.147079446s) [noise=true , address=77.90.31.19:4843 , piece_id=XFDYVPVOBBQCIQMPR4JCLLEKPVGEAM4YJPOSJIHPLC6VDPUROCIA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.128114600 (26h7m42.16768772s) [noise=true , address=45.91.116.8:4048 , piece_id=3I5YOPQ7PTGWH5PAL74QSMLHVKPPX6BWTWTH47YM6QJU33XCEBJQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.265087974 (26h7m42.304661094s) [noise=true , address=77.90.31.16:4734 , piece_id=SP7SAVF572CM5CTE6VYS67HQCAKZQ34J4FRZEWGGNX6CIRKRG53A, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.389194034 (26h7m42.428767144s) [noise=true , address=77.90.31.19:4841 , piece_id=UE2V2DI7EYDJWBPGIEHT2TI22HB22LPITW5FTEXIUJEOTU67GXMQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.389221016 (26h7m42.428794136s) [seg_pos={0 0}, upload=23004]: Segment reached optimal threshold of 13 pieces. Sep 7 15:36:06.389324835 (26h7m42.428897955s) [noise=true , address=45.91.116.7:4011 , piece_id=G7ZYPHTSX4R2EFI5ZZFN7ACVF6E53XT5OLQG2I6VVJEFYXQ3IIKQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Sep 7 15:36:06.389478469 (26h7m42.429051589s) [noise=true , address=77.90.31.13:4628 , piece_id=OZP6LRGQ6WCDLHDVQISDVUEFRZAEUJ3UDCZ36HBLR25O6XBTB2NA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection; write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection Sep 7 15:36:06.389496152 (26h7m42.429069272s) [noise=true , address=45.91.116.12:4226 , piece_id=S6ULWGIZZHY56GQ7LQ47H25NVEBPS3UHU5QYPLUT4BEBZ2GXZXMA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection; write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection Sep 7 15:36:06.389538985 (26h7m42.429112105s) [seg_pos={0 0}, upload=23004]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:36:06.389608707 (26h7m42.429181827s) [upload=23004]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:36:06.389612845 (26h7m42.429185965s) [upload=23004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:36:06.739578335 (26h7m42.779151455s) [upload=23004]: Done uploading segments... Sep 7 15:36:06.739598183 (26h7m42.779171294s) [upload=23004]: Upload finished. err: <nil> Duration: 7.184915276s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [XX......X....X..X........XX.XX..X...............X.......X...] duration: 7.933834357s FAIL (2025-09-07 15:38:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:36:03.192216018 (26h7m39.231789138s) [upload=23004]: Starting upload Sep 7 15:36:03.192274741 (26h7m39.231847861s) [upload=23004]: Done starting upload Sep 7 15:36:03.192338963 (26h7m39.231912083s) [upload=23004]: Uploading segments... Sep 7 15:36:03.192454144 (26h7m39.232027264s) [upload=23004]: Got next segment. Inline: false Sep 7 15:36:03.192482228 (26h7m39.232055338s) [upload=23004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:36:03.192486767 (26h7m39.232059887s) [upload=23004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:36:03.487935637 (26h7m39.527508757s) [seg_pos={0 0}, upload=23004]: Begin upload segment... Sep 7 15:36:03.487980703 (26h7m39.527553823s) [seg_pos={0 0}, upload=23004]: Done begin upload segment. Sep 7 15:36:03.487999539 (26h7m39.527572659s) [noise=true , address=77.90.31.19:4842 , piece_id=3MVZTN47VLDKNXVRTGLAZ3MDLUT3GT4VKM7KMWLWTNJDKIDUHYHA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488113476 (26h7m39.527686596s) [noise=true , address=77.90.31.12:4590 , piece_id=2244PECVOTKU26LAELDDEOFMKAOCJIOPWBEK3IGBE5SB5E2VMMJA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488477183 (26h7m39.528050303s) [noise=true , address=45.91.116.4:4939 , piece_id=QMWLG3GX6PY5IA3PVUDWXHXAA6SRORXKASIHUPDJGBKNQQHEHIPQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488635737 (26h7m39.528208857s) [noise=true , address=45.91.116.7:4011 , piece_id=G7ZYPHTSX4R2EFI5ZZFN7ACVF6E53XT5OLQG2I6VVJEFYXQ3IIKQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488812204 (26h7m39.528385324s) [noise=true , address=77.90.31.10:4518 , piece_id=THTBURY3HOWHHEGR3S6ZRPECXNAO3M5D26ATLHJHZR4SWXWAMY2Q, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488958824 (26h7m39.528531944s) [noise=true , address=45.91.116.9:4261 , piece_id=QDOA5NPWGPWPDH6XZYJD7UAF3TTZHCJI47DVIAPFE6HF3CCF67EQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489097559 (26h7m39.5286707s) [noise=true , address=77.90.31.19:4841 , piece_id=UE2V2DI7EYDJWBPGIEHT2TI22HB22LPITW5FTEXIUJEOTU67GXMQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489302172 (26h7m39.528875292s) [noise=true , address=77.90.31.13:4628 , piece_id=OZP6LRGQ6WCDLHDVQISDVUEFRZAEUJ3UDCZ36HBLR25O6XBTB2NA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489435627 (26h7m39.529008747s) [noise=true , address=45.91.116.8:4048 , piece_id=3I5YOPQ7PTGWH5PAL74QSMLHVKPPX6BWTWTH47YM6QJU33XCEBJQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489574192 (26h7m39.529147312s) [noise=true , address=77.90.31.19:4843 , piece_id=XFDYVPVOBBQCIQMPR4JCLLEKPVGEAM4YJPOSJIHPLC6VDPUROCIA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489661028 (26h7m39.529234148s) [noise=true , address=77.90.31.16:4734 , piece_id=SP7SAVF572CM5CTE6VYS67HQCAKZQ34J4FRZEWGGNX6CIRKRG53A, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489896860 (26h7m39.52946998s) [noise=true , address=45.91.116.13:4154 , piece_id=YFYTQG3ZEYWRBBEMZ2PF4UIUK7VD2YBDP4GWVZL7XS754ZUQQ7SA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490009024 (26h7m39.529582144s) [noise=true , address=45.91.116.14:4189 , piece_id=MHAVVRDB3DC6VZ6NZQKRXEYWGKYWBQXLEGBS73TVEEZASOTT5XRA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490112943 (26h7m39.529686063s) [noise=true , address=77.90.31.14:4663 , piece_id=FEL4QHHHUXPGFYWBFTFMO346E5MEJHYSJIEQEQA6T5D6EUK5GSFA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490277498 (26h7m39.529850618s) [noise=true , address=77.90.31.15:4697 , piece_id=4X2HZTVC24PARUSP7ZFJQQBP644BKJY7ZS6QPONEDFCNKROMENVQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490575549 (26h7m39.530148659s) [noise=true , address=45.91.116.12:4226 , piece_id=S6ULWGIZZHY56GQ7LQ47H25NVEBPS3UHU5QYPLUT4BEBZ2GXZXMA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:05.499052531 (26h7m41.538625651s) [upload=23004]: Next returned nil segment Sep 7 15:36:05.499070735 (26h7m41.538643845s) [upload=23004]: Waiting for error group managing segments... Sep 7 15:36:05.885906268 (26h7m41.925479388s) [noise=true , address=45.91.116.9:4261 , piece_id=QDOA5NPWGPWPDH6XZYJD7UAF3TTZHCJI47DVIAPFE6HF3CCF67EQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.897076963 (26h7m41.936650083s) [noise=true , address=45.91.116.4:4939 , piece_id=QMWLG3GX6PY5IA3PVUDWXHXAA6SRORXKASIHUPDJGBKNQQHEHIPQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.923811300 (26h7m41.96338442s) [noise=true , address=45.91.116.13:4154 , piece_id=YFYTQG3ZEYWRBBEMZ2PF4UIUK7VD2YBDP4GWVZL7XS754ZUQQ7SA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.942724120 (26h7m41.98229723s) [noise=true , address=77.90.31.19:4842 , piece_id=3MVZTN47VLDKNXVRTGLAZ3MDLUT3GT4VKM7KMWLWTNJDKIDUHYHA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.953070108 (26h7m41.992643228s) [noise=true , address=77.90.31.12:4590 , piece_id=2244PECVOTKU26LAELDDEOFMKAOCJIOPWBEK3IGBE5SB5E2VMMJA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.961410377 (26h7m42.000983497s) [noise=true , address=77.90.31.15:4697 , piece_id=4X2HZTVC24PARUSP7ZFJQQBP644BKJY7ZS6QPONEDFCNKROMENVQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.997943784 (26h7m42.037516894s) [noise=true , address=45.91.116.14:4189 , piece_id=MHAVVRDB3DC6VZ6NZQKRXEYWGKYWBQXLEGBS73TVEEZASOTT5XRA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.003770574 (26h7m42.043343694s) [noise=true , address=77.90.31.14:4663 , piece_id=FEL4QHHHUXPGFYWBFTFMO346E5MEJHYSJIEQEQA6T5D6EUK5GSFA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.071696002 (26h7m42.111269122s) [noise=true , address=77.90.31.10:4518 , piece_id=THTBURY3HOWHHEGR3S6ZRPECXNAO3M5D26ATLHJHZR4SWXWAMY2Q, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.107506336 (26h7m42.147079446s) [noise=true , address=77.90.31.19:4843 , piece_id=XFDYVPVOBBQCIQMPR4JCLLEKPVGEAM4YJPOSJIHPLC6VDPUROCIA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.128114600 (26h7m42.16768772s) [noise=true , address=45.91.116.8:4048 , piece_id=3I5YOPQ7PTGWH5PAL74QSMLHVKPPX6BWTWTH47YM6QJU33XCEBJQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.265087974 (26h7m42.304661094s) [noise=true , address=77.90.31.16:4734 , piece_id=SP7SAVF572CM5CTE6VYS67HQCAKZQ34J4FRZEWGGNX6CIRKRG53A, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.389194034 (26h7m42.428767144s) [noise=true , address=77.90.31.19:4841 , piece_id=UE2V2DI7EYDJWBPGIEHT2TI22HB22LPITW5FTEXIUJEOTU67GXMQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.389221016 (26h7m42.428794136s) [seg_pos={0 0}, upload=23004]: Segment reached optimal threshold of 13 pieces. Sep 7 15:36:06.389324835 (26h7m42.428897955s) [noise=true , address=45.91.116.7:4011 , piece_id=G7ZYPHTSX4R2EFI5ZZFN7ACVF6E53XT5OLQG2I6VVJEFYXQ3IIKQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Sep 7 15:36:06.389478469 (26h7m42.429051589s) [noise=true , address=77.90.31.13:4628 , piece_id=OZP6LRGQ6WCDLHDVQISDVUEFRZAEUJ3UDCZ36HBLR25O6XBTB2NA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection; write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection Sep 7 15:36:06.389496152 (26h7m42.429069272s) [noise=true , address=45.91.116.12:4226 , piece_id=S6ULWGIZZHY56GQ7LQ47H25NVEBPS3UHU5QYPLUT4BEBZ2GXZXMA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection; write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection Sep 7 15:36:06.389538985 (26h7m42.429112105s) [seg_pos={0 0}, upload=23004]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:36:06.389608707 (26h7m42.429181827s) [upload=23004]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:36:06.389612845 (26h7m42.429185965s) [upload=23004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:36:06.739578335 (26h7m42.779151455s) [upload=23004]: Done uploading segments... Sep 7 15:36:06.739598183 (26h7m42.779171294s) [upload=23004]: Upload finished. err: <nil> Duration: 7.93386671s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [XXX......X....X..X........XX.XX..X...............X.......X..] duration: 6.413691426s FAIL (2025-09-07 15:37:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:36:03.192216018 (26h7m39.231789138s) [upload=23004]: Starting upload Sep 7 15:36:03.192274741 (26h7m39.231847861s) [upload=23004]: Done starting upload Sep 7 15:36:03.192338963 (26h7m39.231912083s) [upload=23004]: Uploading segments... Sep 7 15:36:03.192454144 (26h7m39.232027264s) [upload=23004]: Got next segment. Inline: false Sep 7 15:36:03.192482228 (26h7m39.232055338s) [upload=23004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:36:03.192486767 (26h7m39.232059887s) [upload=23004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:36:03.487935637 (26h7m39.527508757s) [seg_pos={0 0}, upload=23004]: Begin upload segment... Sep 7 15:36:03.487980703 (26h7m39.527553823s) [seg_pos={0 0}, upload=23004]: Done begin upload segment. Sep 7 15:36:03.487999539 (26h7m39.527572659s) [noise=true , address=77.90.31.19:4842 , piece_id=3MVZTN47VLDKNXVRTGLAZ3MDLUT3GT4VKM7KMWLWTNJDKIDUHYHA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488113476 (26h7m39.527686596s) [noise=true , address=77.90.31.12:4590 , piece_id=2244PECVOTKU26LAELDDEOFMKAOCJIOPWBEK3IGBE5SB5E2VMMJA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488477183 (26h7m39.528050303s) [noise=true , address=45.91.116.4:4939 , piece_id=QMWLG3GX6PY5IA3PVUDWXHXAA6SRORXKASIHUPDJGBKNQQHEHIPQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488635737 (26h7m39.528208857s) [noise=true , address=45.91.116.7:4011 , piece_id=G7ZYPHTSX4R2EFI5ZZFN7ACVF6E53XT5OLQG2I6VVJEFYXQ3IIKQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488812204 (26h7m39.528385324s) [noise=true , address=77.90.31.10:4518 , piece_id=THTBURY3HOWHHEGR3S6ZRPECXNAO3M5D26ATLHJHZR4SWXWAMY2Q, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488958824 (26h7m39.528531944s) [noise=true , address=45.91.116.9:4261 , piece_id=QDOA5NPWGPWPDH6XZYJD7UAF3TTZHCJI47DVIAPFE6HF3CCF67EQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489097559 (26h7m39.5286707s) [noise=true , address=77.90.31.19:4841 , piece_id=UE2V2DI7EYDJWBPGIEHT2TI22HB22LPITW5FTEXIUJEOTU67GXMQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489302172 (26h7m39.528875292s) [noise=true , address=77.90.31.13:4628 , piece_id=OZP6LRGQ6WCDLHDVQISDVUEFRZAEUJ3UDCZ36HBLR25O6XBTB2NA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489435627 (26h7m39.529008747s) [noise=true , address=45.91.116.8:4048 , piece_id=3I5YOPQ7PTGWH5PAL74QSMLHVKPPX6BWTWTH47YM6QJU33XCEBJQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489574192 (26h7m39.529147312s) [noise=true , address=77.90.31.19:4843 , piece_id=XFDYVPVOBBQCIQMPR4JCLLEKPVGEAM4YJPOSJIHPLC6VDPUROCIA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489661028 (26h7m39.529234148s) [noise=true , address=77.90.31.16:4734 , piece_id=SP7SAVF572CM5CTE6VYS67HQCAKZQ34J4FRZEWGGNX6CIRKRG53A, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489896860 (26h7m39.52946998s) [noise=true , address=45.91.116.13:4154 , piece_id=YFYTQG3ZEYWRBBEMZ2PF4UIUK7VD2YBDP4GWVZL7XS754ZUQQ7SA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490009024 (26h7m39.529582144s) [noise=true , address=45.91.116.14:4189 , piece_id=MHAVVRDB3DC6VZ6NZQKRXEYWGKYWBQXLEGBS73TVEEZASOTT5XRA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490112943 (26h7m39.529686063s) [noise=true , address=77.90.31.14:4663 , piece_id=FEL4QHHHUXPGFYWBFTFMO346E5MEJHYSJIEQEQA6T5D6EUK5GSFA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490277498 (26h7m39.529850618s) [noise=true , address=77.90.31.15:4697 , piece_id=4X2HZTVC24PARUSP7ZFJQQBP644BKJY7ZS6QPONEDFCNKROMENVQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490575549 (26h7m39.530148659s) [noise=true , address=45.91.116.12:4226 , piece_id=S6ULWGIZZHY56GQ7LQ47H25NVEBPS3UHU5QYPLUT4BEBZ2GXZXMA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:05.499052531 (26h7m41.538625651s) [upload=23004]: Next returned nil segment Sep 7 15:36:05.499070735 (26h7m41.538643845s) [upload=23004]: Waiting for error group managing segments... Sep 7 15:36:05.885906268 (26h7m41.925479388s) [noise=true , address=45.91.116.9:4261 , piece_id=QDOA5NPWGPWPDH6XZYJD7UAF3TTZHCJI47DVIAPFE6HF3CCF67EQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.897076963 (26h7m41.936650083s) [noise=true , address=45.91.116.4:4939 , piece_id=QMWLG3GX6PY5IA3PVUDWXHXAA6SRORXKASIHUPDJGBKNQQHEHIPQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.923811300 (26h7m41.96338442s) [noise=true , address=45.91.116.13:4154 , piece_id=YFYTQG3ZEYWRBBEMZ2PF4UIUK7VD2YBDP4GWVZL7XS754ZUQQ7SA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.942724120 (26h7m41.98229723s) [noise=true , address=77.90.31.19:4842 , piece_id=3MVZTN47VLDKNXVRTGLAZ3MDLUT3GT4VKM7KMWLWTNJDKIDUHYHA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.953070108 (26h7m41.992643228s) [noise=true , address=77.90.31.12:4590 , piece_id=2244PECVOTKU26LAELDDEOFMKAOCJIOPWBEK3IGBE5SB5E2VMMJA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.961410377 (26h7m42.000983497s) [noise=true , address=77.90.31.15:4697 , piece_id=4X2HZTVC24PARUSP7ZFJQQBP644BKJY7ZS6QPONEDFCNKROMENVQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.997943784 (26h7m42.037516894s) [noise=true , address=45.91.116.14:4189 , piece_id=MHAVVRDB3DC6VZ6NZQKRXEYWGKYWBQXLEGBS73TVEEZASOTT5XRA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.003770574 (26h7m42.043343694s) [noise=true , address=77.90.31.14:4663 , piece_id=FEL4QHHHUXPGFYWBFTFMO346E5MEJHYSJIEQEQA6T5D6EUK5GSFA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.071696002 (26h7m42.111269122s) [noise=true , address=77.90.31.10:4518 , piece_id=THTBURY3HOWHHEGR3S6ZRPECXNAO3M5D26ATLHJHZR4SWXWAMY2Q, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.107506336 (26h7m42.147079446s) [noise=true , address=77.90.31.19:4843 , piece_id=XFDYVPVOBBQCIQMPR4JCLLEKPVGEAM4YJPOSJIHPLC6VDPUROCIA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.128114600 (26h7m42.16768772s) [noise=true , address=45.91.116.8:4048 , piece_id=3I5YOPQ7PTGWH5PAL74QSMLHVKPPX6BWTWTH47YM6QJU33XCEBJQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.265087974 (26h7m42.304661094s) [noise=true , address=77.90.31.16:4734 , piece_id=SP7SAVF572CM5CTE6VYS67HQCAKZQ34J4FRZEWGGNX6CIRKRG53A, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.389194034 (26h7m42.428767144s) [noise=true , address=77.90.31.19:4841 , piece_id=UE2V2DI7EYDJWBPGIEHT2TI22HB22LPITW5FTEXIUJEOTU67GXMQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.389221016 (26h7m42.428794136s) [seg_pos={0 0}, upload=23004]: Segment reached optimal threshold of 13 pieces. Sep 7 15:36:06.389324835 (26h7m42.428897955s) [noise=true , address=45.91.116.7:4011 , piece_id=G7ZYPHTSX4R2EFI5ZZFN7ACVF6E53XT5OLQG2I6VVJEFYXQ3IIKQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Sep 7 15:36:06.389478469 (26h7m42.429051589s) [noise=true , address=77.90.31.13:4628 , piece_id=OZP6LRGQ6WCDLHDVQISDVUEFRZAEUJ3UDCZ36HBLR25O6XBTB2NA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection; write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection Sep 7 15:36:06.389496152 (26h7m42.429069272s) [noise=true , address=45.91.116.12:4226 , piece_id=S6ULWGIZZHY56GQ7LQ47H25NVEBPS3UHU5QYPLUT4BEBZ2GXZXMA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection; write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection Sep 7 15:36:06.389538985 (26h7m42.429112105s) [seg_pos={0 0}, upload=23004]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:36:06.389608707 (26h7m42.429181827s) [upload=23004]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:36:06.389612845 (26h7m42.429185965s) [upload=23004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:36:06.739578335 (26h7m42.779151455s) [upload=23004]: Done uploading segments... Sep 7 15:36:06.739598183 (26h7m42.779171294s) [upload=23004]: Upload finished. err: <nil> Duration: 6.413713048s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.XXX......X....X..X........XX.XX..X...............X.......X.] duration: 11.778959744s FAIL (2025-09-07 15:37:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:36:03.192216018 (26h7m39.231789138s) [upload=23004]: Starting upload Sep 7 15:36:03.192274741 (26h7m39.231847861s) [upload=23004]: Done starting upload Sep 7 15:36:03.192338963 (26h7m39.231912083s) [upload=23004]: Uploading segments... Sep 7 15:36:03.192454144 (26h7m39.232027264s) [upload=23004]: Got next segment. Inline: false Sep 7 15:36:03.192482228 (26h7m39.232055338s) [upload=23004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:36:03.192486767 (26h7m39.232059887s) [upload=23004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:36:03.487935637 (26h7m39.527508757s) [seg_pos={0 0}, upload=23004]: Begin upload segment... Sep 7 15:36:03.487980703 (26h7m39.527553823s) [seg_pos={0 0}, upload=23004]: Done begin upload segment. Sep 7 15:36:03.487999539 (26h7m39.527572659s) [noise=true , address=77.90.31.19:4842 , piece_id=3MVZTN47VLDKNXVRTGLAZ3MDLUT3GT4VKM7KMWLWTNJDKIDUHYHA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488113476 (26h7m39.527686596s) [noise=true , address=77.90.31.12:4590 , piece_id=2244PECVOTKU26LAELDDEOFMKAOCJIOPWBEK3IGBE5SB5E2VMMJA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488477183 (26h7m39.528050303s) [noise=true , address=45.91.116.4:4939 , piece_id=QMWLG3GX6PY5IA3PVUDWXHXAA6SRORXKASIHUPDJGBKNQQHEHIPQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488635737 (26h7m39.528208857s) [noise=true , address=45.91.116.7:4011 , piece_id=G7ZYPHTSX4R2EFI5ZZFN7ACVF6E53XT5OLQG2I6VVJEFYXQ3IIKQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488812204 (26h7m39.528385324s) [noise=true , address=77.90.31.10:4518 , piece_id=THTBURY3HOWHHEGR3S6ZRPECXNAO3M5D26ATLHJHZR4SWXWAMY2Q, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488958824 (26h7m39.528531944s) [noise=true , address=45.91.116.9:4261 , piece_id=QDOA5NPWGPWPDH6XZYJD7UAF3TTZHCJI47DVIAPFE6HF3CCF67EQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489097559 (26h7m39.5286707s) [noise=true , address=77.90.31.19:4841 , piece_id=UE2V2DI7EYDJWBPGIEHT2TI22HB22LPITW5FTEXIUJEOTU67GXMQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489302172 (26h7m39.528875292s) [noise=true , address=77.90.31.13:4628 , piece_id=OZP6LRGQ6WCDLHDVQISDVUEFRZAEUJ3UDCZ36HBLR25O6XBTB2NA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489435627 (26h7m39.529008747s) [noise=true , address=45.91.116.8:4048 , piece_id=3I5YOPQ7PTGWH5PAL74QSMLHVKPPX6BWTWTH47YM6QJU33XCEBJQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489574192 (26h7m39.529147312s) [noise=true , address=77.90.31.19:4843 , piece_id=XFDYVPVOBBQCIQMPR4JCLLEKPVGEAM4YJPOSJIHPLC6VDPUROCIA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489661028 (26h7m39.529234148s) [noise=true , address=77.90.31.16:4734 , piece_id=SP7SAVF572CM5CTE6VYS67HQCAKZQ34J4FRZEWGGNX6CIRKRG53A, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489896860 (26h7m39.52946998s) [noise=true , address=45.91.116.13:4154 , piece_id=YFYTQG3ZEYWRBBEMZ2PF4UIUK7VD2YBDP4GWVZL7XS754ZUQQ7SA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490009024 (26h7m39.529582144s) [noise=true , address=45.91.116.14:4189 , piece_id=MHAVVRDB3DC6VZ6NZQKRXEYWGKYWBQXLEGBS73TVEEZASOTT5XRA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490112943 (26h7m39.529686063s) [noise=true , address=77.90.31.14:4663 , piece_id=FEL4QHHHUXPGFYWBFTFMO346E5MEJHYSJIEQEQA6T5D6EUK5GSFA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490277498 (26h7m39.529850618s) [noise=true , address=77.90.31.15:4697 , piece_id=4X2HZTVC24PARUSP7ZFJQQBP644BKJY7ZS6QPONEDFCNKROMENVQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490575549 (26h7m39.530148659s) [noise=true , address=45.91.116.12:4226 , piece_id=S6ULWGIZZHY56GQ7LQ47H25NVEBPS3UHU5QYPLUT4BEBZ2GXZXMA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:05.499052531 (26h7m41.538625651s) [upload=23004]: Next returned nil segment Sep 7 15:36:05.499070735 (26h7m41.538643845s) [upload=23004]: Waiting for error group managing segments... Sep 7 15:36:05.885906268 (26h7m41.925479388s) [noise=true , address=45.91.116.9:4261 , piece_id=QDOA5NPWGPWPDH6XZYJD7UAF3TTZHCJI47DVIAPFE6HF3CCF67EQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.897076963 (26h7m41.936650083s) [noise=true , address=45.91.116.4:4939 , piece_id=QMWLG3GX6PY5IA3PVUDWXHXAA6SRORXKASIHUPDJGBKNQQHEHIPQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.923811300 (26h7m41.96338442s) [noise=true , address=45.91.116.13:4154 , piece_id=YFYTQG3ZEYWRBBEMZ2PF4UIUK7VD2YBDP4GWVZL7XS754ZUQQ7SA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.942724120 (26h7m41.98229723s) [noise=true , address=77.90.31.19:4842 , piece_id=3MVZTN47VLDKNXVRTGLAZ3MDLUT3GT4VKM7KMWLWTNJDKIDUHYHA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.953070108 (26h7m41.992643228s) [noise=true , address=77.90.31.12:4590 , piece_id=2244PECVOTKU26LAELDDEOFMKAOCJIOPWBEK3IGBE5SB5E2VMMJA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.961410377 (26h7m42.000983497s) [noise=true , address=77.90.31.15:4697 , piece_id=4X2HZTVC24PARUSP7ZFJQQBP644BKJY7ZS6QPONEDFCNKROMENVQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.997943784 (26h7m42.037516894s) [noise=true , address=45.91.116.14:4189 , piece_id=MHAVVRDB3DC6VZ6NZQKRXEYWGKYWBQXLEGBS73TVEEZASOTT5XRA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.003770574 (26h7m42.043343694s) [noise=true , address=77.90.31.14:4663 , piece_id=FEL4QHHHUXPGFYWBFTFMO346E5MEJHYSJIEQEQA6T5D6EUK5GSFA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.071696002 (26h7m42.111269122s) [noise=true , address=77.90.31.10:4518 , piece_id=THTBURY3HOWHHEGR3S6ZRPECXNAO3M5D26ATLHJHZR4SWXWAMY2Q, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.107506336 (26h7m42.147079446s) [noise=true , address=77.90.31.19:4843 , piece_id=XFDYVPVOBBQCIQMPR4JCLLEKPVGEAM4YJPOSJIHPLC6VDPUROCIA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.128114600 (26h7m42.16768772s) [noise=true , address=45.91.116.8:4048 , piece_id=3I5YOPQ7PTGWH5PAL74QSMLHVKPPX6BWTWTH47YM6QJU33XCEBJQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.265087974 (26h7m42.304661094s) [noise=true , address=77.90.31.16:4734 , piece_id=SP7SAVF572CM5CTE6VYS67HQCAKZQ34J4FRZEWGGNX6CIRKRG53A, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.389194034 (26h7m42.428767144s) [noise=true , address=77.90.31.19:4841 , piece_id=UE2V2DI7EYDJWBPGIEHT2TI22HB22LPITW5FTEXIUJEOTU67GXMQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.389221016 (26h7m42.428794136s) [seg_pos={0 0}, upload=23004]: Segment reached optimal threshold of 13 pieces. Sep 7 15:36:06.389324835 (26h7m42.428897955s) [noise=true , address=45.91.116.7:4011 , piece_id=G7ZYPHTSX4R2EFI5ZZFN7ACVF6E53XT5OLQG2I6VVJEFYXQ3IIKQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Sep 7 15:36:06.389478469 (26h7m42.429051589s) [noise=true , address=77.90.31.13:4628 , piece_id=OZP6LRGQ6WCDLHDVQISDVUEFRZAEUJ3UDCZ36HBLR25O6XBTB2NA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection; write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection Sep 7 15:36:06.389496152 (26h7m42.429069272s) [noise=true , address=45.91.116.12:4226 , piece_id=S6ULWGIZZHY56GQ7LQ47H25NVEBPS3UHU5QYPLUT4BEBZ2GXZXMA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection; write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection Sep 7 15:36:06.389538985 (26h7m42.429112105s) [seg_pos={0 0}, upload=23004]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:36:06.389608707 (26h7m42.429181827s) [upload=23004]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:36:06.389612845 (26h7m42.429185965s) [upload=23004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:36:06.739578335 (26h7m42.779151455s) [upload=23004]: Done uploading segments... Sep 7 15:36:06.739598183 (26h7m42.779171294s) [upload=23004]: Upload finished. err: <nil> Duration: 11.778994131s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [X.XXX......X....X..X........XX.XX..X...............X.......X] duration: 30.00215082s FAIL (2025-09-07 15:36:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:36:03.192216018 (26h7m39.231789138s) [upload=23004]: Starting upload Sep 7 15:36:03.192274741 (26h7m39.231847861s) [upload=23004]: Done starting upload Sep 7 15:36:03.192338963 (26h7m39.231912083s) [upload=23004]: Uploading segments... Sep 7 15:36:03.192454144 (26h7m39.232027264s) [upload=23004]: Got next segment. Inline: false Sep 7 15:36:03.192482228 (26h7m39.232055338s) [upload=23004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:36:03.192486767 (26h7m39.232059887s) [upload=23004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:36:03.487935637 (26h7m39.527508757s) [seg_pos={0 0}, upload=23004]: Begin upload segment... Sep 7 15:36:03.487980703 (26h7m39.527553823s) [seg_pos={0 0}, upload=23004]: Done begin upload segment. Sep 7 15:36:03.487999539 (26h7m39.527572659s) [noise=true , address=77.90.31.19:4842 , piece_id=3MVZTN47VLDKNXVRTGLAZ3MDLUT3GT4VKM7KMWLWTNJDKIDUHYHA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488113476 (26h7m39.527686596s) [noise=true , address=77.90.31.12:4590 , piece_id=2244PECVOTKU26LAELDDEOFMKAOCJIOPWBEK3IGBE5SB5E2VMMJA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488477183 (26h7m39.528050303s) [noise=true , address=45.91.116.4:4939 , piece_id=QMWLG3GX6PY5IA3PVUDWXHXAA6SRORXKASIHUPDJGBKNQQHEHIPQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488635737 (26h7m39.528208857s) [noise=true , address=45.91.116.7:4011 , piece_id=G7ZYPHTSX4R2EFI5ZZFN7ACVF6E53XT5OLQG2I6VVJEFYXQ3IIKQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488812204 (26h7m39.528385324s) [noise=true , address=77.90.31.10:4518 , piece_id=THTBURY3HOWHHEGR3S6ZRPECXNAO3M5D26ATLHJHZR4SWXWAMY2Q, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.488958824 (26h7m39.528531944s) [noise=true , address=45.91.116.9:4261 , piece_id=QDOA5NPWGPWPDH6XZYJD7UAF3TTZHCJI47DVIAPFE6HF3CCF67EQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489097559 (26h7m39.5286707s) [noise=true , address=77.90.31.19:4841 , piece_id=UE2V2DI7EYDJWBPGIEHT2TI22HB22LPITW5FTEXIUJEOTU67GXMQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489302172 (26h7m39.528875292s) [noise=true , address=77.90.31.13:4628 , piece_id=OZP6LRGQ6WCDLHDVQISDVUEFRZAEUJ3UDCZ36HBLR25O6XBTB2NA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489435627 (26h7m39.529008747s) [noise=true , address=45.91.116.8:4048 , piece_id=3I5YOPQ7PTGWH5PAL74QSMLHVKPPX6BWTWTH47YM6QJU33XCEBJQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489574192 (26h7m39.529147312s) [noise=true , address=77.90.31.19:4843 , piece_id=XFDYVPVOBBQCIQMPR4JCLLEKPVGEAM4YJPOSJIHPLC6VDPUROCIA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489661028 (26h7m39.529234148s) [noise=true , address=77.90.31.16:4734 , piece_id=SP7SAVF572CM5CTE6VYS67HQCAKZQ34J4FRZEWGGNX6CIRKRG53A, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.489896860 (26h7m39.52946998s) [noise=true , address=45.91.116.13:4154 , piece_id=YFYTQG3ZEYWRBBEMZ2PF4UIUK7VD2YBDP4GWVZL7XS754ZUQQ7SA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490009024 (26h7m39.529582144s) [noise=true , address=45.91.116.14:4189 , piece_id=MHAVVRDB3DC6VZ6NZQKRXEYWGKYWBQXLEGBS73TVEEZASOTT5XRA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490112943 (26h7m39.529686063s) [noise=true , address=77.90.31.14:4663 , piece_id=FEL4QHHHUXPGFYWBFTFMO346E5MEJHYSJIEQEQA6T5D6EUK5GSFA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490277498 (26h7m39.529850618s) [noise=true , address=77.90.31.15:4697 , piece_id=4X2HZTVC24PARUSP7ZFJQQBP644BKJY7ZS6QPONEDFCNKROMENVQ, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:03.490575549 (26h7m39.530148659s) [noise=true , address=45.91.116.12:4226 , piece_id=S6ULWGIZZHY56GQ7LQ47H25NVEBPS3UHU5QYPLUT4BEBZ2GXZXMA, seg_pos={0 0}, upload=23004]: Uploading piece... Sep 7 15:36:05.499052531 (26h7m41.538625651s) [upload=23004]: Next returned nil segment Sep 7 15:36:05.499070735 (26h7m41.538643845s) [upload=23004]: Waiting for error group managing segments... Sep 7 15:36:05.885906268 (26h7m41.925479388s) [noise=true , address=45.91.116.9:4261 , piece_id=QDOA5NPWGPWPDH6XZYJD7UAF3TTZHCJI47DVIAPFE6HF3CCF67EQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.897076963 (26h7m41.936650083s) [noise=true , address=45.91.116.4:4939 , piece_id=QMWLG3GX6PY5IA3PVUDWXHXAA6SRORXKASIHUPDJGBKNQQHEHIPQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.923811300 (26h7m41.96338442s) [noise=true , address=45.91.116.13:4154 , piece_id=YFYTQG3ZEYWRBBEMZ2PF4UIUK7VD2YBDP4GWVZL7XS754ZUQQ7SA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.942724120 (26h7m41.98229723s) [noise=true , address=77.90.31.19:4842 , piece_id=3MVZTN47VLDKNXVRTGLAZ3MDLUT3GT4VKM7KMWLWTNJDKIDUHYHA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.953070108 (26h7m41.992643228s) [noise=true , address=77.90.31.12:4590 , piece_id=2244PECVOTKU26LAELDDEOFMKAOCJIOPWBEK3IGBE5SB5E2VMMJA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.961410377 (26h7m42.000983497s) [noise=true , address=77.90.31.15:4697 , piece_id=4X2HZTVC24PARUSP7ZFJQQBP644BKJY7ZS6QPONEDFCNKROMENVQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:05.997943784 (26h7m42.037516894s) [noise=true , address=45.91.116.14:4189 , piece_id=MHAVVRDB3DC6VZ6NZQKRXEYWGKYWBQXLEGBS73TVEEZASOTT5XRA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.003770574 (26h7m42.043343694s) [noise=true , address=77.90.31.14:4663 , piece_id=FEL4QHHHUXPGFYWBFTFMO346E5MEJHYSJIEQEQA6T5D6EUK5GSFA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.071696002 (26h7m42.111269122s) [noise=true , address=77.90.31.10:4518 , piece_id=THTBURY3HOWHHEGR3S6ZRPECXNAO3M5D26ATLHJHZR4SWXWAMY2Q, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.107506336 (26h7m42.147079446s) [noise=true , address=77.90.31.19:4843 , piece_id=XFDYVPVOBBQCIQMPR4JCLLEKPVGEAM4YJPOSJIHPLC6VDPUROCIA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.128114600 (26h7m42.16768772s) [noise=true , address=45.91.116.8:4048 , piece_id=3I5YOPQ7PTGWH5PAL74QSMLHVKPPX6BWTWTH47YM6QJU33XCEBJQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.265087974 (26h7m42.304661094s) [noise=true , address=77.90.31.16:4734 , piece_id=SP7SAVF572CM5CTE6VYS67HQCAKZQ34J4FRZEWGGNX6CIRKRG53A, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.389194034 (26h7m42.428767144s) [noise=true , address=77.90.31.19:4841 , piece_id=UE2V2DI7EYDJWBPGIEHT2TI22HB22LPITW5FTEXIUJEOTU67GXMQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: <nil> Sep 7 15:36:06.389221016 (26h7m42.428794136s) [seg_pos={0 0}, upload=23004]: Segment reached optimal threshold of 13 pieces. Sep 7 15:36:06.389324835 (26h7m42.428897955s) [noise=true , address=45.91.116.7:4011 , piece_id=G7ZYPHTSX4R2EFI5ZZFN7ACVF6E53XT5OLQG2I6VVJEFYXQ3IIKQ, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Sep 7 15:36:06.389478469 (26h7m42.429051589s) [noise=true , address=77.90.31.13:4628 , piece_id=OZP6LRGQ6WCDLHDVQISDVUEFRZAEUJ3UDCZ36HBLR25O6XBTB2NA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection; write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43986->77.90.31.13:4628: use of closed network connection Sep 7 15:36:06.389496152 (26h7m42.429069272s) [noise=true , address=45.91.116.12:4226 , piece_id=S6ULWGIZZHY56GQ7LQ47H25NVEBPS3UHU5QYPLUT4BEBZ2GXZXMA, seg_pos={0 0}, upload=23004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection; write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42718->45.91.116.12:4226: use of closed network connection Sep 7 15:36:06.389538985 (26h7m42.429112105s) [seg_pos={0 0}, upload=23004]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:36:06.389608707 (26h7m42.429181827s) [upload=23004]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:36:06.389612845 (26h7m42.429185965s) [upload=23004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:36:06.739578335 (26h7m42.779151455s) [upload=23004]: Done uploading segments... Sep 7 15:36:06.739598183 (26h7m42.779171294s) [upload=23004]: Upload finished. err: <nil> Duration: 30.002181257s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.X.XXX......X....X..X........XX.XX..X...............X.......] duration: 8.259621209s FAIL (2025-09-07 15:35:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:29:31.109150726 (26h1m7.148723836s) [upload=22908]: Starting upload Sep 7 15:29:31.109205461 (26h1m7.148778581s) [upload=22908]: Done starting upload Sep 7 15:29:31.109331793 (26h1m7.148904913s) [upload=22908]: Uploading segments... Sep 7 15:29:31.109420983 (26h1m7.148994103s) [upload=22908]: Got next segment. Inline: false Sep 7 15:29:31.109460649 (26h1m7.149033759s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:29:31.109464856 (26h1m7.149037967s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:29:31.405827016 (26h1m7.445400136s) [seg_pos={0 0}, upload=22908]: Begin upload segment... Sep 7 15:29:31.405863706 (26h1m7.445436826s) [seg_pos={0 0}, upload=22908]: Done begin upload segment. Sep 7 15:29:31.405883995 (26h1m7.445457115s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.405932837 (26h1m7.445505957s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406248412 (26h1m7.445821532s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406414159 (26h1m7.445987279s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406529840 (26h1m7.44610295s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406681430 (26h1m7.446254551s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406784919 (26h1m7.446358029s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407019257 (26h1m7.446592377s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407258715 (26h1m7.446831835s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407353085 (26h1m7.446926205s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407585741 (26h1m7.447158861s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407685422 (26h1m7.447258552s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407812154 (26h1m7.447385274s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408108040 (26h1m7.44768116s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408569092 (26h1m7.448142213s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408905766 (26h1m7.448478886s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:33.245130862 (26h1m9.284703982s) [upload=22908]: Next returned nil segment Sep 7 15:29:33.245148185 (26h1m9.284721296s) [upload=22908]: Waiting for error group managing segments... Sep 7 15:29:33.575725105 (26h1m9.615298225s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.590608014 (26h1m9.630181134s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.596425456 (26h1m9.635998576s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.601155567 (26h1m9.640728687s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.613073782 (26h1m9.652646902s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.616754566 (26h1m9.656327686s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.619808580 (26h1m9.65938169s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.624328138 (26h1m9.663901258s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.655884332 (26h1m9.695457452s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.656812508 (26h1m9.696385628s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.678366562 (26h1m9.717939683s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.737959024 (26h1m9.777532144s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753513980 (26h1m9.7930871s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753534328 (26h1m9.793107448s) [seg_pos={0 0}, upload=22908]: Segment reached optimal threshold of 13 pieces. Sep 7 15:29:33.753653156 (26h1m9.793226286s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753711878 (26h1m9.793284998s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753846967 (26h1m9.793420087s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753898105 (26h1m9.793471225s) [seg_pos={0 0}, upload=22908]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:29:33.754007364 (26h1m9.793580484s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:29:33.754013375 (26h1m9.793586535s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:29:34.165145340 (26h1m10.20471845s) [upload=22908]: Done uploading segments... Sep 7 15:29:34.165162021 (26h1m10.204735131s) [upload=22908]: Upload finished. err: <nil> Duration: 8.259659402s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [X.X.XXX......X....X..X........XX.XX..X...............X......] duration: 14.837257103s FAIL (2025-09-07 15:34:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:29:31.109150726 (26h1m7.148723836s) [upload=22908]: Starting upload Sep 7 15:29:31.109205461 (26h1m7.148778581s) [upload=22908]: Done starting upload Sep 7 15:29:31.109331793 (26h1m7.148904913s) [upload=22908]: Uploading segments... Sep 7 15:29:31.109420983 (26h1m7.148994103s) [upload=22908]: Got next segment. Inline: false Sep 7 15:29:31.109460649 (26h1m7.149033759s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:29:31.109464856 (26h1m7.149037967s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:29:31.405827016 (26h1m7.445400136s) [seg_pos={0 0}, upload=22908]: Begin upload segment... Sep 7 15:29:31.405863706 (26h1m7.445436826s) [seg_pos={0 0}, upload=22908]: Done begin upload segment. Sep 7 15:29:31.405883995 (26h1m7.445457115s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.405932837 (26h1m7.445505957s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406248412 (26h1m7.445821532s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406414159 (26h1m7.445987279s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406529840 (26h1m7.44610295s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406681430 (26h1m7.446254551s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406784919 (26h1m7.446358029s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407019257 (26h1m7.446592377s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407258715 (26h1m7.446831835s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407353085 (26h1m7.446926205s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407585741 (26h1m7.447158861s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407685422 (26h1m7.447258552s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407812154 (26h1m7.447385274s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408108040 (26h1m7.44768116s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408569092 (26h1m7.448142213s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408905766 (26h1m7.448478886s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:33.245130862 (26h1m9.284703982s) [upload=22908]: Next returned nil segment Sep 7 15:29:33.245148185 (26h1m9.284721296s) [upload=22908]: Waiting for error group managing segments... Sep 7 15:29:33.575725105 (26h1m9.615298225s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.590608014 (26h1m9.630181134s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.596425456 (26h1m9.635998576s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.601155567 (26h1m9.640728687s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.613073782 (26h1m9.652646902s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.616754566 (26h1m9.656327686s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.619808580 (26h1m9.65938169s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.624328138 (26h1m9.663901258s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.655884332 (26h1m9.695457452s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.656812508 (26h1m9.696385628s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.678366562 (26h1m9.717939683s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.737959024 (26h1m9.777532144s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753513980 (26h1m9.7930871s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753534328 (26h1m9.793107448s) [seg_pos={0 0}, upload=22908]: Segment reached optimal threshold of 13 pieces. Sep 7 15:29:33.753653156 (26h1m9.793226286s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753711878 (26h1m9.793284998s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753846967 (26h1m9.793420087s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753898105 (26h1m9.793471225s) [seg_pos={0 0}, upload=22908]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:29:33.754007364 (26h1m9.793580484s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:29:33.754013375 (26h1m9.793586535s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:29:34.165145340 (26h1m10.20471845s) [upload=22908]: Done uploading segments... Sep 7 15:29:34.165162021 (26h1m10.204735131s) [upload=22908]: Upload finished. err: <nil> Duration: 14.837288031s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [XX.X.XXX......X....X..X........XX.XX..X...............X.....] duration: 6.455352142s FAIL (2025-09-07 15:33:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:29:31.109150726 (26h1m7.148723836s) [upload=22908]: Starting upload Sep 7 15:29:31.109205461 (26h1m7.148778581s) [upload=22908]: Done starting upload Sep 7 15:29:31.109331793 (26h1m7.148904913s) [upload=22908]: Uploading segments... Sep 7 15:29:31.109420983 (26h1m7.148994103s) [upload=22908]: Got next segment. Inline: false Sep 7 15:29:31.109460649 (26h1m7.149033759s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:29:31.109464856 (26h1m7.149037967s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:29:31.405827016 (26h1m7.445400136s) [seg_pos={0 0}, upload=22908]: Begin upload segment... Sep 7 15:29:31.405863706 (26h1m7.445436826s) [seg_pos={0 0}, upload=22908]: Done begin upload segment. Sep 7 15:29:31.405883995 (26h1m7.445457115s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.405932837 (26h1m7.445505957s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406248412 (26h1m7.445821532s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406414159 (26h1m7.445987279s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406529840 (26h1m7.44610295s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406681430 (26h1m7.446254551s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406784919 (26h1m7.446358029s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407019257 (26h1m7.446592377s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407258715 (26h1m7.446831835s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407353085 (26h1m7.446926205s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407585741 (26h1m7.447158861s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407685422 (26h1m7.447258552s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407812154 (26h1m7.447385274s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408108040 (26h1m7.44768116s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408569092 (26h1m7.448142213s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408905766 (26h1m7.448478886s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:33.245130862 (26h1m9.284703982s) [upload=22908]: Next returned nil segment Sep 7 15:29:33.245148185 (26h1m9.284721296s) [upload=22908]: Waiting for error group managing segments... Sep 7 15:29:33.575725105 (26h1m9.615298225s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.590608014 (26h1m9.630181134s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.596425456 (26h1m9.635998576s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.601155567 (26h1m9.640728687s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.613073782 (26h1m9.652646902s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.616754566 (26h1m9.656327686s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.619808580 (26h1m9.65938169s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.624328138 (26h1m9.663901258s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.655884332 (26h1m9.695457452s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.656812508 (26h1m9.696385628s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.678366562 (26h1m9.717939683s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.737959024 (26h1m9.777532144s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753513980 (26h1m9.7930871s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753534328 (26h1m9.793107448s) [seg_pos={0 0}, upload=22908]: Segment reached optimal threshold of 13 pieces. Sep 7 15:29:33.753653156 (26h1m9.793226286s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753711878 (26h1m9.793284998s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753846967 (26h1m9.793420087s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753898105 (26h1m9.793471225s) [seg_pos={0 0}, upload=22908]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:29:33.754007364 (26h1m9.793580484s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:29:33.754013375 (26h1m9.793586535s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:29:34.165145340 (26h1m10.20471845s) [upload=22908]: Done uploading segments... Sep 7 15:29:34.165162021 (26h1m10.204735131s) [upload=22908]: Upload finished. err: <nil> Duration: 6.455384854s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.XX.X.XXX......X....X..X........XX.XX..X...............X....] duration: 15.571829039s FAIL (2025-09-07 15:32:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:29:31.109150726 (26h1m7.148723836s) [upload=22908]: Starting upload Sep 7 15:29:31.109205461 (26h1m7.148778581s) [upload=22908]: Done starting upload Sep 7 15:29:31.109331793 (26h1m7.148904913s) [upload=22908]: Uploading segments... Sep 7 15:29:31.109420983 (26h1m7.148994103s) [upload=22908]: Got next segment. Inline: false Sep 7 15:29:31.109460649 (26h1m7.149033759s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:29:31.109464856 (26h1m7.149037967s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:29:31.405827016 (26h1m7.445400136s) [seg_pos={0 0}, upload=22908]: Begin upload segment... Sep 7 15:29:31.405863706 (26h1m7.445436826s) [seg_pos={0 0}, upload=22908]: Done begin upload segment. Sep 7 15:29:31.405883995 (26h1m7.445457115s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.405932837 (26h1m7.445505957s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406248412 (26h1m7.445821532s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406414159 (26h1m7.445987279s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406529840 (26h1m7.44610295s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406681430 (26h1m7.446254551s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406784919 (26h1m7.446358029s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407019257 (26h1m7.446592377s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407258715 (26h1m7.446831835s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407353085 (26h1m7.446926205s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407585741 (26h1m7.447158861s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407685422 (26h1m7.447258552s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407812154 (26h1m7.447385274s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408108040 (26h1m7.44768116s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408569092 (26h1m7.448142213s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408905766 (26h1m7.448478886s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:33.245130862 (26h1m9.284703982s) [upload=22908]: Next returned nil segment Sep 7 15:29:33.245148185 (26h1m9.284721296s) [upload=22908]: Waiting for error group managing segments... Sep 7 15:29:33.575725105 (26h1m9.615298225s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.590608014 (26h1m9.630181134s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.596425456 (26h1m9.635998576s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.601155567 (26h1m9.640728687s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.613073782 (26h1m9.652646902s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.616754566 (26h1m9.656327686s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.619808580 (26h1m9.65938169s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.624328138 (26h1m9.663901258s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.655884332 (26h1m9.695457452s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.656812508 (26h1m9.696385628s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.678366562 (26h1m9.717939683s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.737959024 (26h1m9.777532144s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753513980 (26h1m9.7930871s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753534328 (26h1m9.793107448s) [seg_pos={0 0}, upload=22908]: Segment reached optimal threshold of 13 pieces. Sep 7 15:29:33.753653156 (26h1m9.793226286s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753711878 (26h1m9.793284998s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753846967 (26h1m9.793420087s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753898105 (26h1m9.793471225s) [seg_pos={0 0}, upload=22908]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:29:33.754007364 (26h1m9.793580484s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:29:33.754013375 (26h1m9.793586535s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:29:34.165145340 (26h1m10.20471845s) [upload=22908]: Done uploading segments... Sep 7 15:29:34.165162021 (26h1m10.204735131s) [upload=22908]: Upload finished. err: <nil> Duration: 15.571858285s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [..XX.X.XXX......X....X..X........XX.XX..X...............X...] duration: 8.277612544s FAIL (2025-09-07 15:31:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:29:31.109150726 (26h1m7.148723836s) [upload=22908]: Starting upload Sep 7 15:29:31.109205461 (26h1m7.148778581s) [upload=22908]: Done starting upload Sep 7 15:29:31.109331793 (26h1m7.148904913s) [upload=22908]: Uploading segments... Sep 7 15:29:31.109420983 (26h1m7.148994103s) [upload=22908]: Got next segment. Inline: false Sep 7 15:29:31.109460649 (26h1m7.149033759s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:29:31.109464856 (26h1m7.149037967s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:29:31.405827016 (26h1m7.445400136s) [seg_pos={0 0}, upload=22908]: Begin upload segment... Sep 7 15:29:31.405863706 (26h1m7.445436826s) [seg_pos={0 0}, upload=22908]: Done begin upload segment. Sep 7 15:29:31.405883995 (26h1m7.445457115s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.405932837 (26h1m7.445505957s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406248412 (26h1m7.445821532s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406414159 (26h1m7.445987279s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406529840 (26h1m7.44610295s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406681430 (26h1m7.446254551s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406784919 (26h1m7.446358029s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407019257 (26h1m7.446592377s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407258715 (26h1m7.446831835s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407353085 (26h1m7.446926205s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407585741 (26h1m7.447158861s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407685422 (26h1m7.447258552s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407812154 (26h1m7.447385274s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408108040 (26h1m7.44768116s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408569092 (26h1m7.448142213s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408905766 (26h1m7.448478886s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:33.245130862 (26h1m9.284703982s) [upload=22908]: Next returned nil segment Sep 7 15:29:33.245148185 (26h1m9.284721296s) [upload=22908]: Waiting for error group managing segments... Sep 7 15:29:33.575725105 (26h1m9.615298225s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.590608014 (26h1m9.630181134s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.596425456 (26h1m9.635998576s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.601155567 (26h1m9.640728687s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.613073782 (26h1m9.652646902s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.616754566 (26h1m9.656327686s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.619808580 (26h1m9.65938169s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.624328138 (26h1m9.663901258s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.655884332 (26h1m9.695457452s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.656812508 (26h1m9.696385628s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.678366562 (26h1m9.717939683s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.737959024 (26h1m9.777532144s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753513980 (26h1m9.7930871s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753534328 (26h1m9.793107448s) [seg_pos={0 0}, upload=22908]: Segment reached optimal threshold of 13 pieces. Sep 7 15:29:33.753653156 (26h1m9.793226286s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753711878 (26h1m9.793284998s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753846967 (26h1m9.793420087s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753898105 (26h1m9.793471225s) [seg_pos={0 0}, upload=22908]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:29:33.754007364 (26h1m9.793580484s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:29:33.754013375 (26h1m9.793586535s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:29:34.165145340 (26h1m10.20471845s) [upload=22908]: Done uploading segments... Sep 7 15:29:34.165162021 (26h1m10.204735131s) [upload=22908]: Upload finished. err: <nil> Duration: 8.277754426s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [...XX.X.XXX......X....X..X........XX.XX..X...............X..] duration: 12.910779535s FAIL (2025-09-07 15:30:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:29:31.109150726 (26h1m7.148723836s) [upload=22908]: Starting upload Sep 7 15:29:31.109205461 (26h1m7.148778581s) [upload=22908]: Done starting upload Sep 7 15:29:31.109331793 (26h1m7.148904913s) [upload=22908]: Uploading segments... Sep 7 15:29:31.109420983 (26h1m7.148994103s) [upload=22908]: Got next segment. Inline: false Sep 7 15:29:31.109460649 (26h1m7.149033759s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:29:31.109464856 (26h1m7.149037967s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:29:31.405827016 (26h1m7.445400136s) [seg_pos={0 0}, upload=22908]: Begin upload segment... Sep 7 15:29:31.405863706 (26h1m7.445436826s) [seg_pos={0 0}, upload=22908]: Done begin upload segment. Sep 7 15:29:31.405883995 (26h1m7.445457115s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.405932837 (26h1m7.445505957s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406248412 (26h1m7.445821532s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406414159 (26h1m7.445987279s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406529840 (26h1m7.44610295s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406681430 (26h1m7.446254551s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406784919 (26h1m7.446358029s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407019257 (26h1m7.446592377s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407258715 (26h1m7.446831835s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407353085 (26h1m7.446926205s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407585741 (26h1m7.447158861s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407685422 (26h1m7.447258552s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407812154 (26h1m7.447385274s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408108040 (26h1m7.44768116s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408569092 (26h1m7.448142213s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408905766 (26h1m7.448478886s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:33.245130862 (26h1m9.284703982s) [upload=22908]: Next returned nil segment Sep 7 15:29:33.245148185 (26h1m9.284721296s) [upload=22908]: Waiting for error group managing segments... Sep 7 15:29:33.575725105 (26h1m9.615298225s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.590608014 (26h1m9.630181134s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.596425456 (26h1m9.635998576s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.601155567 (26h1m9.640728687s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.613073782 (26h1m9.652646902s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.616754566 (26h1m9.656327686s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.619808580 (26h1m9.65938169s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.624328138 (26h1m9.663901258s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.655884332 (26h1m9.695457452s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.656812508 (26h1m9.696385628s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.678366562 (26h1m9.717939683s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.737959024 (26h1m9.777532144s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753513980 (26h1m9.7930871s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753534328 (26h1m9.793107448s) [seg_pos={0 0}, upload=22908]: Segment reached optimal threshold of 13 pieces. Sep 7 15:29:33.753653156 (26h1m9.793226286s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753711878 (26h1m9.793284998s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753846967 (26h1m9.793420087s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753898105 (26h1m9.793471225s) [seg_pos={0 0}, upload=22908]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:29:33.754007364 (26h1m9.793580484s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:29:33.754013375 (26h1m9.793586535s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:29:34.165145340 (26h1m10.20471845s) [upload=22908]: Done uploading segments... Sep 7 15:29:34.165162021 (26h1m10.204735131s) [upload=22908]: Upload finished. err: <nil> Duration: 12.910808521s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [....XX.X.XXX......X....X..X........XX.XX..X...............X.] duration: 6.481170735s FAIL (2025-09-07 15:30:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:29:31.109150726 (26h1m7.148723836s) [upload=22908]: Starting upload Sep 7 15:29:31.109205461 (26h1m7.148778581s) [upload=22908]: Done starting upload Sep 7 15:29:31.109331793 (26h1m7.148904913s) [upload=22908]: Uploading segments... Sep 7 15:29:31.109420983 (26h1m7.148994103s) [upload=22908]: Got next segment. Inline: false Sep 7 15:29:31.109460649 (26h1m7.149033759s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:29:31.109464856 (26h1m7.149037967s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:29:31.405827016 (26h1m7.445400136s) [seg_pos={0 0}, upload=22908]: Begin upload segment... Sep 7 15:29:31.405863706 (26h1m7.445436826s) [seg_pos={0 0}, upload=22908]: Done begin upload segment. Sep 7 15:29:31.405883995 (26h1m7.445457115s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.405932837 (26h1m7.445505957s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406248412 (26h1m7.445821532s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406414159 (26h1m7.445987279s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406529840 (26h1m7.44610295s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406681430 (26h1m7.446254551s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406784919 (26h1m7.446358029s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407019257 (26h1m7.446592377s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407258715 (26h1m7.446831835s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407353085 (26h1m7.446926205s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407585741 (26h1m7.447158861s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407685422 (26h1m7.447258552s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407812154 (26h1m7.447385274s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408108040 (26h1m7.44768116s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408569092 (26h1m7.448142213s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408905766 (26h1m7.448478886s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:33.245130862 (26h1m9.284703982s) [upload=22908]: Next returned nil segment Sep 7 15:29:33.245148185 (26h1m9.284721296s) [upload=22908]: Waiting for error group managing segments... Sep 7 15:29:33.575725105 (26h1m9.615298225s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.590608014 (26h1m9.630181134s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.596425456 (26h1m9.635998576s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.601155567 (26h1m9.640728687s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.613073782 (26h1m9.652646902s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.616754566 (26h1m9.656327686s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.619808580 (26h1m9.65938169s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.624328138 (26h1m9.663901258s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.655884332 (26h1m9.695457452s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.656812508 (26h1m9.696385628s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.678366562 (26h1m9.717939683s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.737959024 (26h1m9.777532144s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753513980 (26h1m9.7930871s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753534328 (26h1m9.793107448s) [seg_pos={0 0}, upload=22908]: Segment reached optimal threshold of 13 pieces. Sep 7 15:29:33.753653156 (26h1m9.793226286s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753711878 (26h1m9.793284998s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753846967 (26h1m9.793420087s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753898105 (26h1m9.793471225s) [seg_pos={0 0}, upload=22908]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:29:33.754007364 (26h1m9.793580484s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:29:33.754013375 (26h1m9.793586535s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:29:34.165145340 (26h1m10.20471845s) [upload=22908]: Done uploading segments... Sep 7 15:29:34.165162021 (26h1m10.204735131s) [upload=22908]: Upload finished. err: <nil> Duration: 6.481187136s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.....XX.X.XXX......X....X..X........XX.XX..X...............X] duration: 30.002460977s FAIL (2025-09-07 15:30:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:29:31.109150726 (26h1m7.148723836s) [upload=22908]: Starting upload Sep 7 15:29:31.109205461 (26h1m7.148778581s) [upload=22908]: Done starting upload Sep 7 15:29:31.109331793 (26h1m7.148904913s) [upload=22908]: Uploading segments... Sep 7 15:29:31.109420983 (26h1m7.148994103s) [upload=22908]: Got next segment. Inline: false Sep 7 15:29:31.109460649 (26h1m7.149033759s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:29:31.109464856 (26h1m7.149037967s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:29:31.405827016 (26h1m7.445400136s) [seg_pos={0 0}, upload=22908]: Begin upload segment... Sep 7 15:29:31.405863706 (26h1m7.445436826s) [seg_pos={0 0}, upload=22908]: Done begin upload segment. Sep 7 15:29:31.405883995 (26h1m7.445457115s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.405932837 (26h1m7.445505957s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406248412 (26h1m7.445821532s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406414159 (26h1m7.445987279s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406529840 (26h1m7.44610295s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406681430 (26h1m7.446254551s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.406784919 (26h1m7.446358029s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407019257 (26h1m7.446592377s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407258715 (26h1m7.446831835s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407353085 (26h1m7.446926205s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407585741 (26h1m7.447158861s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407685422 (26h1m7.447258552s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.407812154 (26h1m7.447385274s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408108040 (26h1m7.44768116s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408569092 (26h1m7.448142213s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:31.408905766 (26h1m7.448478886s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Uploading piece... Sep 7 15:29:33.245130862 (26h1m9.284703982s) [upload=22908]: Next returned nil segment Sep 7 15:29:33.245148185 (26h1m9.284721296s) [upload=22908]: Waiting for error group managing segments... Sep 7 15:29:33.575725105 (26h1m9.615298225s) [noise=true , address=77.90.31.10:4519 , piece_id=BSD5PLZVYDOSZRLUJC62PEVS6FWF6UIHX6VCOV6TX6S5NKGX7RJQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.590608014 (26h1m9.630181134s) [noise=true , address=45.91.116.5:4299 , piece_id=HFZQH6WOEW6WSSOVVREI4BD3ZDQVV5SL2S5PWENZZGZB6TTHBNWA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.596425456 (26h1m9.635998576s) [noise=true , address=77.90.31.12:4592 , piece_id=BYVLYKNK56VENW4NBV7MY7CGKIIYXZ6H7WPREMHPRHADZZA6E2SQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.601155567 (26h1m9.640728687s) [noise=true , address=45.91.116.7:4009 , piece_id=YQLJGLN2T22SWPXUU6KX5S4TXOJZZR7C5MO6NITO43U5ONVA3LYA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.613073782 (26h1m9.652646902s) [noise=true , address=45.91.116.10:4083 , piece_id=RITW2XIK6KQOQTX2PQEVO6FSLYNLCLR7PFYOYKBBD46FUQWKCZIQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.616754566 (26h1m9.656327686s) [noise=true , address=45.91.116.5:4297 , piece_id=VO24IYPOMGWYUSZ3SM4GT5HE4RV5JKBXDCHCIV5HOAZ5V647JCAA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.619808580 (26h1m9.65938169s) [noise=true , address=45.91.116.10:4081 , piece_id=MR2ALHN3R5AEY4WXWN3QD2ICPUQ6QSWM3LSFWJRU5V2COS2M3QCA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.624328138 (26h1m9.663901258s) [noise=true , address=77.90.31.14:4662 , piece_id=CX6AONYKLLEVLH6QYUX7AZVYMJJCDCSRTMRTBVOMNA7VFYDIKDMQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.655884332 (26h1m9.695457452s) [noise=true , address=45.91.116.8:4045 , piece_id=SQVPRF6U7U47PILNKPADCFFLQZPQNES6ZMZ4IVCGGEMYB35AS6BA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.656812508 (26h1m9.696385628s) [noise=true , address=77.90.31.15:4697 , piece_id=2EFE6XTNASWUDZ75AZVZ2MNX2RQTPUOA3EKGUJFCTB2NFHWRJKDQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.678366562 (26h1m9.717939683s) [noise=true , address=77.90.31.16:4733 , piece_id=UYYK2MYYPGDNCIEQEP7LML4L4IRJ7PK2YHCVWH6QPPN3ETCSMPEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.737959024 (26h1m9.777532144s) [noise=true , address=45.91.116.14:4191 , piece_id=PMFHTVDEXTK4H3OVDUFFAO4SAQ4BFWL4TD6YOX7OHLBHGK6EJUFQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753513980 (26h1m9.7930871s) [noise=true , address=77.90.31.18:4807 , piece_id=RFLEGN7GHUVH7YTVP5M5A2GT72QHB7HBY7NRH5MURSVW3FGLYBEQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: <nil> Sep 7 15:29:33.753534328 (26h1m9.793107448s) [seg_pos={0 0}, upload=22908]: Segment reached optimal threshold of 13 pieces. Sep 7 15:29:33.753653156 (26h1m9.793226286s) [noise=true , address=77.90.31.21:4879 , piece_id=2YDWMEWYZXZVE4JNNHYY2HWQVJQNWGFZOL2SE746U2KVHSR44KNA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753711878 (26h1m9.793284998s) [noise=true , address=77.90.31.13:4626 , piece_id=Q6LI6BWUZCW66OE4TKCZLM737ZLDEYWRSRZOD64KYWFGYSD2GSAQ, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753846967 (26h1m9.793420087s) [noise=true , address=45.91.116.12:4226 , piece_id=XEFV2NJAOTTIM4TK7QODCEZRFHO45D4PNQDOAPWJHOSMXJWTLBGA, seg_pos={0 0}, upload=22908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:29:33.753898105 (26h1m9.793471225s) [seg_pos={0 0}, upload=22908]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:29:33.754007364 (26h1m9.793580484s) [upload=22908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:29:33.754013375 (26h1m9.793586535s) [upload=22908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:29:34.165145340 (26h1m10.20471845s) [upload=22908]: Done uploading segments... Sep 7 15:29:34.165162021 (26h1m10.204735131s) [upload=22908]: Upload finished. err: <nil> Duration: 30.002481436s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [......XX.X.XXX......X....X..X........XX.XX..X...............] duration: 6.980844719s FAIL (2025-09-07 15:28:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 6.980868995s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.......XX.X.XXX......X....X..X........XX.XX..X..............] duration: 10.595305964s FAIL (2025-09-07 15:28:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 10.595325922s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [X.......XX.X.XXX......X....X..X........XX.XX..X.............] duration: 6.621827737s FAIL (2025-09-07 15:27:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 6.621853896s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.X.......XX.X.XXX......X....X..X........XX.XX..X............] duration: 12.714160059s FAIL (2025-09-07 15:26:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 12.714187392s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [..X.......XX.X.XXX......X....X..X........XX.XX..X...........] duration: 15.050180397s FAIL (2025-09-07 15:26:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 15.05020754s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [X..X.......XX.X.XXX......X....X..X........XX.XX..X..........] duration: 15.203528476s FAIL (2025-09-07 15:25:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 15.203553553s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [XX..X.......XX.X.XXX......X....X..X........XX.XX..X.........] duration: 12.871343222s FAIL (2025-09-07 15:24:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 12.871362109s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.XX..X.......XX.X.XXX......X....X..X........XX.XX..X........] duration: 11.191446509s FAIL (2025-09-07 15:24:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 11.191462169s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [..XX..X.......XX.X.XXX......X....X..X........XX.XX..X.......] duration: 9.282825019s FAIL (2025-09-07 15:23:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 9.282848985s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [X..XX..X.......XX.X.XXX......X....X..X........XX.XX..X......] duration: 8.94238083s FAIL (2025-09-07 15:23:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 8.942399806s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [.X..XX..X.......XX.X.XXX......X....X..X........XX.XX..X.....] duration: 17.808182172s FAIL (2025-09-07 15:22:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 17.808211248s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [X.X..XX..X.......XX.X.XXX......X....X..X........XX.XX..X....] duration: 16.322672701s FAIL (2025-09-07 15:21:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 16.322700523s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [XX.X..XX..X.......XX.X.XXX......X....X..X........XX.XX..X...] duration: 9.04814346s FAIL (2025-09-07 15:21:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 9.048169871s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [XXX.X..XX..X.......XX.X.XXX......X....X..X........XX.XX..X..] duration: 7.100946817s FAIL (2025-09-07 15:20:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 7.100962809s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [XXXX.X..XX..X.......XX.X.XXX......X....X..X........XX.XX..X.] duration: 29.998536158s FAIL (2025-09-07 15:20:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 29.99856371s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [.XXXX.X..XX..X.......XX.X.XXX......X....X..X........XX.XX..X] duration: 30.002688727s FAIL (2025-09-07 15:19:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:19:07.655278034 (25h50m43.694851175s) [upload=22748]: Starting upload Sep 7 15:19:07.655416018 (25h50m43.694989189s) [upload=22748]: Done starting upload Sep 7 15:19:07.655623605 (25h50m43.695196755s) [upload=22748]: Uploading segments... Sep 7 15:19:07.655862212 (25h50m43.695435372s) [upload=22748]: Got next segment. Inline: false Sep 7 15:19:07.655922758 (25h50m43.695495908s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:19:07.655931545 (25h50m43.695504695s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:19:07.924610751 (25h50m43.964183871s) [seg_pos={0 0}, upload=22748]: Begin upload segment... Sep 7 15:19:07.924646990 (25h50m43.96422011s) [seg_pos={0 0}, upload=22748]: Done begin upload segment. Sep 7 15:19:07.924675435 (25h50m43.964248545s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.924688059 (25h50m43.964261169s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925036796 (25h50m43.964609906s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925242079 (25h50m43.964815199s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925312213 (25h50m43.964885333s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925564146 (25h50m43.965137266s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925612257 (25h50m43.965185377s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925904697 (25h50m43.965477807s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.925937089 (25h50m43.965510209s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926162501 (25h50m43.965735621s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926167520 (25h50m43.96574064s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926383424 (25h50m43.965956534s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926410806 (25h50m43.965983926s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926603114 (25h50m43.966176234s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.926835017 (25h50m43.966408137s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:07.928876213 (25h50m43.968449333s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Uploading piece... Sep 7 15:19:10.320146815 (25h50m46.359719935s) [upload=22748]: Next returned nil segment Sep 7 15:19:10.320165221 (25h50m46.359738341s) [upload=22748]: Waiting for error group managing segments... Sep 7 15:19:10.847719953 (25h50m46.887293073s) [noise=true , address=77.90.31.19:4842 , piece_id=YAIP36HCL7XU3DGLEGS42JTXX6RJ45B77VH7MPWMAP56E5BP7MIA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.941629893 (25h50m46.981203023s) [noise=true , address=77.90.31.17:4772 , piece_id=M5Z7CJKZQOULUG4A4XC3BC7X3FMTKUHFXZ4WAWAY6R4MTW733LFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:10.952927482 (25h50m46.992500602s) [noise=true , address=77.90.31.18:4807 , piece_id=H6UZOF36TQWFAFLGMMLYJLWNCJWTVL7IN4UHK7KHDIJOW7XFFWAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.012825438 (25h50m47.052398558s) [noise=true , address=45.91.116.13:4153 , piece_id=B4R3TYOOKNVXCSLJO4MLYRCLXZI64SPPIRMU7MKE7EBEZDVVJJ3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.085082264 (25h50m47.124655384s) [noise=true , address=77.90.31.19:4841 , piece_id=IATTWEZCECJBHXIBBWGOQA2A53Q7XK26VQPKPURQWJQ263GYGJHQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.128016650 (25h50m47.16758977s) [noise=true , address=45.91.116.5:4299 , piece_id=JKZMZC7ELJKJ2YL57C23G2U2FMSBPEZBWFHE6CRGWYLOSB6YL3OA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.143827475 (25h50m47.183400595s) [noise=true , address=45.91.116.14:4189 , piece_id=B3F753QTG6A46YS32GB7VPI6ESWFWZEWRWPKI73SAMAPAZ77KWZQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.348133039 (25h50m47.387706159s) [noise=true , address=45.91.116.6:4973 , piece_id=GNOHSAXBAT6WO2AOGSVBLEZI6QJQBIDRJWXS5OC6W3I24JLU3SUA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.423109930 (25h50m47.46268305s) [noise=true , address=45.91.116.8:4048 , piece_id=MIEKPV25B7AGVFVKWMFEZL3UVKODMTTLOPAK3MJYTCD54DRP6Q3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.494894482 (25h50m47.534467592s) [noise=true , address=77.90.31.10:4520 , piece_id=24PDWJXU7FL3QOGHJ357Q2UNR6BGHJ27HPT6CBCPG27ENPGXWQ3A, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.564275394 (25h50m47.603848514s) [noise=true , address=77.90.31.13:4628 , piece_id=SWRKYO34FDCBIVNWK73U7BRL4CO3IB2HFKV54O3KA74F5VDVDPQQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.612905971 (25h50m47.652479091s) [noise=true , address=77.90.31.12:4590 , piece_id=JPIWEJQI5555MOBTJ6P6WMHR7MW33TQFZ2JR7HXPP2UD23K64YCQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630438401 (25h50m47.670011521s) [noise=true , address=45.91.116.10:4083 , piece_id=N63RUKPSI5NGPYG33YROXLZXKAAMH3F4WXSQQ3XJILOY2BAOQJAQ, seg_pos={0 0}, upload=22748]: Done uploading piece. err: <nil> Sep 7 15:19:11.630478587 (25h50m47.670051707s) [seg_pos={0 0}, upload=22748]: Segment reached optimal threshold of 13 pieces. Sep 7 15:19:11.630701964 (25h50m47.670275084s) [noise=true , address=77.90.31.17:4770 , piece_id=35KVDUCWWEM5SNRXTN3ST6LCOBWZZLFK4N2ORERG2YLJIAJ72Q3Q, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630736391 (25h50m47.670309551s) [noise=true , address=77.90.31.14:4661 , piece_id=YNEWKRMVKDMQYUUOFITPNAHK4NDK3WBJVFOBDXEPCGASZYTYXLFA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630867231 (25h50m47.670440411s) [noise=true , address=77.90.31.16:4734 , piece_id=P4OQCZZE6NGWV3YHKRPME23H7LXWKGUHZFGHQ5BAILTJZJE2NJTA, seg_pos={0 0}, upload=22748]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF Sep 7 15:19:11.630889954 (25h50m47.670463074s) [seg_pos={0 0}, upload=22748]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:19:11.631051283 (25h50m47.670624393s) [upload=22748]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:19:11.631055481 (25h50m47.670628601s) [upload=22748]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:19:11.989436786 (25h50m48.029009896s) [upload=22748]: Done uploading segments... Sep 7 15:19:11.989454790 (25h50m48.02902791s) [upload=22748]: Upload finished. err: <nil> Duration: 30.002719968s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [..XXXX.X..XX..X.......XX.X.XXX......X....X..X........XX.XX..] duration: 15.800935162s FAIL (2025-09-07 15:18:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:16:37.184028564 (25h48m13.223601684s) [upload=22709]: Starting upload Sep 7 15:16:37.184084501 (25h48m13.223657621s) [upload=22709]: Done starting upload Sep 7 15:16:37.184222876 (25h48m13.223795996s) [upload=22709]: Uploading segments... Sep 7 15:16:37.184350130 (25h48m13.22392325s) [upload=22709]: Got next segment. Inline: false Sep 7 15:16:37.184393081 (25h48m13.223966201s) [upload=22709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:16:37.184399384 (25h48m13.223972524s) [upload=22709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:16:37.449287315 (25h48m13.488860435s) [seg_pos={0 0}, upload=22709]: Begin upload segment... Sep 7 15:16:37.449342561 (25h48m13.488915681s) [seg_pos={0 0}, upload=22709]: Done begin upload segment. Sep 7 15:16:37.449364362 (25h48m13.488937472s) [noise=true , address=77.90.31.10:4520 , piece_id=TM7MPCW7DYBNOAPNPZA2B3WX3CKNBAYN2MYSWW7VK7WC7SGZI73Q, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.449932159 (25h48m13.489505279s) [noise=true , address=45.91.116.6:4975 , piece_id=N7HWJUH7YWTW3XNSWFTOCS2WH6LF2PXV2QVNOONVLGTKOJY6YN3A, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450037080 (25h48m13.4896102s) [noise=true , address=77.90.31.16:4733 , piece_id=NB2PS6XLXXOWFSID4DXLXHAF26NFDR3G4TSJL5XCIRF5VAASNF4A, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450214720 (25h48m13.48978784s) [noise=true , address=77.90.31.15:4699 , piece_id=4J5X5BKJZ5EBCVOOBIHGOYGY72QQBJ4J7SKMVHNT4NXLWASSV23A, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450481570 (25h48m13.49005468s) [noise=true , address=77.90.31.21:4877 , piece_id=LECKDTF6AQRU7V2RH3FE5M4KHFHTGKSDYJMIWUHE6PMVA3HN43MQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450504173 (25h48m13.490077293s) [noise=true , address=45.91.116.10:4081 , piece_id=47OA7ZWX46W2BB5TT5WYZIY6BUYPD72IMVK2O3BNRYU2NCTOLBGA, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450765895 (25h48m13.490339015s) [noise=true , address=45.91.116.11:4119 , piece_id=VWNMGBBFCJPXETKVVU56SZSVZO6OKGEOZDSKAME4P7PYVX7J5O3A, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450773038 (25h48m13.490346158s) [noise=true , address=77.90.31.12:4589 , piece_id=L3X22HGVNCLIHIQLLETIYTLWEHZ6B5AQEPAKAQVJFTNQHUMQM5SQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.451114431 (25h48m13.490687551s) [noise=true , address=77.90.31.10:4517 , piece_id=WUSH4DCQXP6P62QHMGSY3BZXXI2GBYCDFXIU7LA2BGT4ICVSJFRQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.451274297 (25h48m13.490847417s) [noise=true , address=45.91.116.6:4973 , piece_id=IZ3ENT4QFABYZ57LFPVD5CYAV56WQTUBRSDUKVXTHRKBO4DNDJMQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.451394547 (25h48m13.490967657s) [noise=true , address=45.91.116.5:4298 , piece_id=DYQ2KJHRRCIUNGX445B6VJMM4VYBUWR5AXAQZ5NYZ34L7QHUGPUQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.451599790 (25h48m13.49117291s) [noise=true , address=45.91.116.13:4153 , piece_id=NLDDITOBVEU6H3HUYUXLM6L5CWR3DUWTUNOPLVJIEKQBNUTMZ5CA, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.451705983 (25h48m13.491279093s) [noise=true , address=77.90.31.12:4592 , piece_id=YCTZJCCSJF25EHDO4P2IE2Z3HHFEOGRFLTOJKW7WUMFBTDMKWPQQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.452034712 (25h48m13.491607832s) [noise=true , address=45.91.116.10:4083 , piece_id=OMUTFXOXUGZ6E57CMXXL6Y2SZVV5PKKLF22DBGBDAXESUQGFX5BQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.452285782 (25h48m13.491858902s) [noise=true , address=77.90.31.14:4663 , piece_id=3HGB2PS4NQTKCNFBG5CBLHLDJPQRPSZ37HPWV4YFP75KU22PMNKQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.452516314 (25h48m13.492089434s) [noise=true , address=77.90.31.13:4626 , piece_id=7NQ6UCG27KACEDWPNAONERF47BQTDX5QF7ATL4QEX7ZIPGVSAVXA, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:39.209584838 (25h48m15.249157958s) [upload=22709]: Next returned nil segment Sep 7 15:16:39.209601631 (25h48m15.249174751s) [upload=22709]: Waiting for error group managing segments... Sep 7 15:16:39.645102354 (25h48m15.684675474s) [noise=true , address=45.91.116.11:4119 , piece_id=VWNMGBBFCJPXETKVVU56SZSVZO6OKGEOZDSKAME4P7PYVX7J5O3A, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.650210057 (25h48m15.689783177s) [noise=true , address=45.91.116.13:4153 , piece_id=NLDDITOBVEU6H3HUYUXLM6L5CWR3DUWTUNOPLVJIEKQBNUTMZ5CA, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.664060150 (25h48m15.70363326s) [noise=true , address=45.91.116.10:4081 , piece_id=47OA7ZWX46W2BB5TT5WYZIY6BUYPD72IMVK2O3BNRYU2NCTOLBGA, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.665130097 (25h48m15.704703217s) [noise=true , address=77.90.31.13:4626 , piece_id=7NQ6UCG27KACEDWPNAONERF47BQTDX5QF7ATL4QEX7ZIPGVSAVXA, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.665736809 (25h48m15.705309929s) [noise=true , address=45.91.116.6:4973 , piece_id=IZ3ENT4QFABYZ57LFPVD5CYAV56WQTUBRSDUKVXTHRKBO4DNDJMQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.669793902 (25h48m15.709367022s) [noise=true , address=45.91.116.6:4975 , piece_id=N7HWJUH7YWTW3XNSWFTOCS2WH6LF2PXV2QVNOONVLGTKOJY6YN3A, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.685929598 (25h48m15.725502718s) [noise=true , address=77.90.31.12:4589 , piece_id=L3X22HGVNCLIHIQLLETIYTLWEHZ6B5AQEPAKAQVJFTNQHUMQM5SQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.687531002 (25h48m15.727104122s) [noise=true , address=77.90.31.16:4733 , piece_id=NB2PS6XLXXOWFSID4DXLXHAF26NFDR3G4TSJL5XCIRF5VAASNF4A, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.741425211 (25h48m15.780998331s) [noise=true , address=77.90.31.21:4877 , piece_id=LECKDTF6AQRU7V2RH3FE5M4KHFHTGKSDYJMIWUHE6PMVA3HN43MQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.749466489 (25h48m15.789039609s) [noise=true , address=77.90.31.12:4592 , piece_id=YCTZJCCSJF25EHDO4P2IE2Z3HHFEOGRFLTOJKW7WUMFBTDMKWPQQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.807397822 (25h48m15.846970942s) [noise=true , address=45.91.116.5:4298 , piece_id=DYQ2KJHRRCIUNGX445B6VJMM4VYBUWR5AXAQZ5NYZ34L7QHUGPUQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.815668978 (25h48m15.855242098s) [noise=true , address=77.90.31.15:4699 , piece_id=4J5X5BKJZ5EBCVOOBIHGOYGY72QQBJ4J7SKMVHNT4NXLWASSV23A, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.854077455 (25h48m15.893650575s) [noise=true , address=45.91.116.10:4083 , piece_id=OMUTFXOXUGZ6E57CMXXL6Y2SZVV5PKKLF22DBGBDAXESUQGFX5BQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.854106962 (25h48m15.893680082s) [seg_pos={0 0}, upload=22709]: Segment reached optimal threshold of 13 pieces. Sep 7 15:16:39.854276235 (25h48m15.893849355s) [noise=true , address=77.90.31.10:4520 , piece_id=TM7MPCW7DYBNOAPNPZA2B3WX3CKNBAYN2MYSWW7VK7WC7SGZI73Q, seg_pos={0 0}, upload=22709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Sep 7 15:16:39.854361609 (25h48m15.893934729s) [noise=true , address=77.90.31.10:4517 , piece_id=WUSH4DCQXP6P62QHMGSY3BZXXI2GBYCDFXIU7LA2BGT4ICVSJFRQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Sep 7 15:16:39.854387168 (25h48m15.893960288s) [noise=true , address=77.90.31.14:4663 , piece_id=3HGB2PS4NQTKCNFBG5CBLHLDJPQRPSZ37HPWV4YFP75KU22PMNKQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:35442->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:35442->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35442->77.90.31.14:4663: use of closed network connection Sep 7 15:16:39.854404782 (25h48m15.893977902s) [seg_pos={0 0}, upload=22709]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:16:39.854505804 (25h48m15.894078924s) [upload=22709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:16:39.854510523 (25h48m15.894083643s) [upload=22709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:16:40.295498076 (25h48m16.335071196s) [upload=22709]: Done uploading segments... Sep 7 15:16:40.295521171 (25h48m16.335094291s) [upload=22709]: Upload finished. err: <nil> Duration: 15.800966032s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [...XXXX.X..XX..X.......XX.X.XXX......X....X..X........XX.XX.] duration: 15.982017722s FAIL (2025-09-07 15:18:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:16:37.184028564 (25h48m13.223601684s) [upload=22709]: Starting upload Sep 7 15:16:37.184084501 (25h48m13.223657621s) [upload=22709]: Done starting upload Sep 7 15:16:37.184222876 (25h48m13.223795996s) [upload=22709]: Uploading segments... Sep 7 15:16:37.184350130 (25h48m13.22392325s) [upload=22709]: Got next segment. Inline: false Sep 7 15:16:37.184393081 (25h48m13.223966201s) [upload=22709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:16:37.184399384 (25h48m13.223972524s) [upload=22709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:16:37.449287315 (25h48m13.488860435s) [seg_pos={0 0}, upload=22709]: Begin upload segment... Sep 7 15:16:37.449342561 (25h48m13.488915681s) [seg_pos={0 0}, upload=22709]: Done begin upload segment. Sep 7 15:16:37.449364362 (25h48m13.488937472s) [noise=true , address=77.90.31.10:4520 , piece_id=TM7MPCW7DYBNOAPNPZA2B3WX3CKNBAYN2MYSWW7VK7WC7SGZI73Q, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.449932159 (25h48m13.489505279s) [noise=true , address=45.91.116.6:4975 , piece_id=N7HWJUH7YWTW3XNSWFTOCS2WH6LF2PXV2QVNOONVLGTKOJY6YN3A, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450037080 (25h48m13.4896102s) [noise=true , address=77.90.31.16:4733 , piece_id=NB2PS6XLXXOWFSID4DXLXHAF26NFDR3G4TSJL5XCIRF5VAASNF4A, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450214720 (25h48m13.48978784s) [noise=true , address=77.90.31.15:4699 , piece_id=4J5X5BKJZ5EBCVOOBIHGOYGY72QQBJ4J7SKMVHNT4NXLWASSV23A, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450481570 (25h48m13.49005468s) [noise=true , address=77.90.31.21:4877 , piece_id=LECKDTF6AQRU7V2RH3FE5M4KHFHTGKSDYJMIWUHE6PMVA3HN43MQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450504173 (25h48m13.490077293s) [noise=true , address=45.91.116.10:4081 , piece_id=47OA7ZWX46W2BB5TT5WYZIY6BUYPD72IMVK2O3BNRYU2NCTOLBGA, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450765895 (25h48m13.490339015s) [noise=true , address=45.91.116.11:4119 , piece_id=VWNMGBBFCJPXETKVVU56SZSVZO6OKGEOZDSKAME4P7PYVX7J5O3A, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450773038 (25h48m13.490346158s) [noise=true , address=77.90.31.12:4589 , piece_id=L3X22HGVNCLIHIQLLETIYTLWEHZ6B5AQEPAKAQVJFTNQHUMQM5SQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.451114431 (25h48m13.490687551s) [noise=true , address=77.90.31.10:4517 , piece_id=WUSH4DCQXP6P62QHMGSY3BZXXI2GBYCDFXIU7LA2BGT4ICVSJFRQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.451274297 (25h48m13.490847417s) [noise=true , address=45.91.116.6:4973 , piece_id=IZ3ENT4QFABYZ57LFPVD5CYAV56WQTUBRSDUKVXTHRKBO4DNDJMQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.451394547 (25h48m13.490967657s) [noise=true , address=45.91.116.5:4298 , piece_id=DYQ2KJHRRCIUNGX445B6VJMM4VYBUWR5AXAQZ5NYZ34L7QHUGPUQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.451599790 (25h48m13.49117291s) [noise=true , address=45.91.116.13:4153 , piece_id=NLDDITOBVEU6H3HUYUXLM6L5CWR3DUWTUNOPLVJIEKQBNUTMZ5CA, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.451705983 (25h48m13.491279093s) [noise=true , address=77.90.31.12:4592 , piece_id=YCTZJCCSJF25EHDO4P2IE2Z3HHFEOGRFLTOJKW7WUMFBTDMKWPQQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.452034712 (25h48m13.491607832s) [noise=true , address=45.91.116.10:4083 , piece_id=OMUTFXOXUGZ6E57CMXXL6Y2SZVV5PKKLF22DBGBDAXESUQGFX5BQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.452285782 (25h48m13.491858902s) [noise=true , address=77.90.31.14:4663 , piece_id=3HGB2PS4NQTKCNFBG5CBLHLDJPQRPSZ37HPWV4YFP75KU22PMNKQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.452516314 (25h48m13.492089434s) [noise=true , address=77.90.31.13:4626 , piece_id=7NQ6UCG27KACEDWPNAONERF47BQTDX5QF7ATL4QEX7ZIPGVSAVXA, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:39.209584838 (25h48m15.249157958s) [upload=22709]: Next returned nil segment Sep 7 15:16:39.209601631 (25h48m15.249174751s) [upload=22709]: Waiting for error group managing segments... Sep 7 15:16:39.645102354 (25h48m15.684675474s) [noise=true , address=45.91.116.11:4119 , piece_id=VWNMGBBFCJPXETKVVU56SZSVZO6OKGEOZDSKAME4P7PYVX7J5O3A, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.650210057 (25h48m15.689783177s) [noise=true , address=45.91.116.13:4153 , piece_id=NLDDITOBVEU6H3HUYUXLM6L5CWR3DUWTUNOPLVJIEKQBNUTMZ5CA, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.664060150 (25h48m15.70363326s) [noise=true , address=45.91.116.10:4081 , piece_id=47OA7ZWX46W2BB5TT5WYZIY6BUYPD72IMVK2O3BNRYU2NCTOLBGA, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.665130097 (25h48m15.704703217s) [noise=true , address=77.90.31.13:4626 , piece_id=7NQ6UCG27KACEDWPNAONERF47BQTDX5QF7ATL4QEX7ZIPGVSAVXA, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.665736809 (25h48m15.705309929s) [noise=true , address=45.91.116.6:4973 , piece_id=IZ3ENT4QFABYZ57LFPVD5CYAV56WQTUBRSDUKVXTHRKBO4DNDJMQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.669793902 (25h48m15.709367022s) [noise=true , address=45.91.116.6:4975 , piece_id=N7HWJUH7YWTW3XNSWFTOCS2WH6LF2PXV2QVNOONVLGTKOJY6YN3A, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.685929598 (25h48m15.725502718s) [noise=true , address=77.90.31.12:4589 , piece_id=L3X22HGVNCLIHIQLLETIYTLWEHZ6B5AQEPAKAQVJFTNQHUMQM5SQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.687531002 (25h48m15.727104122s) [noise=true , address=77.90.31.16:4733 , piece_id=NB2PS6XLXXOWFSID4DXLXHAF26NFDR3G4TSJL5XCIRF5VAASNF4A, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.741425211 (25h48m15.780998331s) [noise=true , address=77.90.31.21:4877 , piece_id=LECKDTF6AQRU7V2RH3FE5M4KHFHTGKSDYJMIWUHE6PMVA3HN43MQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.749466489 (25h48m15.789039609s) [noise=true , address=77.90.31.12:4592 , piece_id=YCTZJCCSJF25EHDO4P2IE2Z3HHFEOGRFLTOJKW7WUMFBTDMKWPQQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.807397822 (25h48m15.846970942s) [noise=true , address=45.91.116.5:4298 , piece_id=DYQ2KJHRRCIUNGX445B6VJMM4VYBUWR5AXAQZ5NYZ34L7QHUGPUQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.815668978 (25h48m15.855242098s) [noise=true , address=77.90.31.15:4699 , piece_id=4J5X5BKJZ5EBCVOOBIHGOYGY72QQBJ4J7SKMVHNT4NXLWASSV23A, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.854077455 (25h48m15.893650575s) [noise=true , address=45.91.116.10:4083 , piece_id=OMUTFXOXUGZ6E57CMXXL6Y2SZVV5PKKLF22DBGBDAXESUQGFX5BQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.854106962 (25h48m15.893680082s) [seg_pos={0 0}, upload=22709]: Segment reached optimal threshold of 13 pieces. Sep 7 15:16:39.854276235 (25h48m15.893849355s) [noise=true , address=77.90.31.10:4520 , piece_id=TM7MPCW7DYBNOAPNPZA2B3WX3CKNBAYN2MYSWW7VK7WC7SGZI73Q, seg_pos={0 0}, upload=22709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Sep 7 15:16:39.854361609 (25h48m15.893934729s) [noise=true , address=77.90.31.10:4517 , piece_id=WUSH4DCQXP6P62QHMGSY3BZXXI2GBYCDFXIU7LA2BGT4ICVSJFRQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Sep 7 15:16:39.854387168 (25h48m15.893960288s) [noise=true , address=77.90.31.14:4663 , piece_id=3HGB2PS4NQTKCNFBG5CBLHLDJPQRPSZ37HPWV4YFP75KU22PMNKQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:35442->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:35442->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35442->77.90.31.14:4663: use of closed network connection Sep 7 15:16:39.854404782 (25h48m15.893977902s) [seg_pos={0 0}, upload=22709]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:16:39.854505804 (25h48m15.894078924s) [upload=22709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:16:39.854510523 (25h48m15.894083643s) [upload=22709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:16:40.295498076 (25h48m16.335071196s) [upload=22709]: Done uploading segments... Sep 7 15:16:40.295521171 (25h48m16.335094291s) [upload=22709]: Upload finished. err: <nil> Duration: 15.982045354s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [....XXXX.X..XX..X.......XX.X.XXX......X....X..X........XX.XX] duration: 30.001682778s FAIL (2025-09-07 15:17:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:16:37.184028564 (25h48m13.223601684s) [upload=22709]: Starting upload Sep 7 15:16:37.184084501 (25h48m13.223657621s) [upload=22709]: Done starting upload Sep 7 15:16:37.184222876 (25h48m13.223795996s) [upload=22709]: Uploading segments... Sep 7 15:16:37.184350130 (25h48m13.22392325s) [upload=22709]: Got next segment. Inline: false Sep 7 15:16:37.184393081 (25h48m13.223966201s) [upload=22709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:16:37.184399384 (25h48m13.223972524s) [upload=22709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:16:37.449287315 (25h48m13.488860435s) [seg_pos={0 0}, upload=22709]: Begin upload segment... Sep 7 15:16:37.449342561 (25h48m13.488915681s) [seg_pos={0 0}, upload=22709]: Done begin upload segment. Sep 7 15:16:37.449364362 (25h48m13.488937472s) [noise=true , address=77.90.31.10:4520 , piece_id=TM7MPCW7DYBNOAPNPZA2B3WX3CKNBAYN2MYSWW7VK7WC7SGZI73Q, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.449932159 (25h48m13.489505279s) [noise=true , address=45.91.116.6:4975 , piece_id=N7HWJUH7YWTW3XNSWFTOCS2WH6LF2PXV2QVNOONVLGTKOJY6YN3A, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450037080 (25h48m13.4896102s) [noise=true , address=77.90.31.16:4733 , piece_id=NB2PS6XLXXOWFSID4DXLXHAF26NFDR3G4TSJL5XCIRF5VAASNF4A, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450214720 (25h48m13.48978784s) [noise=true , address=77.90.31.15:4699 , piece_id=4J5X5BKJZ5EBCVOOBIHGOYGY72QQBJ4J7SKMVHNT4NXLWASSV23A, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450481570 (25h48m13.49005468s) [noise=true , address=77.90.31.21:4877 , piece_id=LECKDTF6AQRU7V2RH3FE5M4KHFHTGKSDYJMIWUHE6PMVA3HN43MQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450504173 (25h48m13.490077293s) [noise=true , address=45.91.116.10:4081 , piece_id=47OA7ZWX46W2BB5TT5WYZIY6BUYPD72IMVK2O3BNRYU2NCTOLBGA, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450765895 (25h48m13.490339015s) [noise=true , address=45.91.116.11:4119 , piece_id=VWNMGBBFCJPXETKVVU56SZSVZO6OKGEOZDSKAME4P7PYVX7J5O3A, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.450773038 (25h48m13.490346158s) [noise=true , address=77.90.31.12:4589 , piece_id=L3X22HGVNCLIHIQLLETIYTLWEHZ6B5AQEPAKAQVJFTNQHUMQM5SQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.451114431 (25h48m13.490687551s) [noise=true , address=77.90.31.10:4517 , piece_id=WUSH4DCQXP6P62QHMGSY3BZXXI2GBYCDFXIU7LA2BGT4ICVSJFRQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.451274297 (25h48m13.490847417s) [noise=true , address=45.91.116.6:4973 , piece_id=IZ3ENT4QFABYZ57LFPVD5CYAV56WQTUBRSDUKVXTHRKBO4DNDJMQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.451394547 (25h48m13.490967657s) [noise=true , address=45.91.116.5:4298 , piece_id=DYQ2KJHRRCIUNGX445B6VJMM4VYBUWR5AXAQZ5NYZ34L7QHUGPUQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.451599790 (25h48m13.49117291s) [noise=true , address=45.91.116.13:4153 , piece_id=NLDDITOBVEU6H3HUYUXLM6L5CWR3DUWTUNOPLVJIEKQBNUTMZ5CA, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.451705983 (25h48m13.491279093s) [noise=true , address=77.90.31.12:4592 , piece_id=YCTZJCCSJF25EHDO4P2IE2Z3HHFEOGRFLTOJKW7WUMFBTDMKWPQQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.452034712 (25h48m13.491607832s) [noise=true , address=45.91.116.10:4083 , piece_id=OMUTFXOXUGZ6E57CMXXL6Y2SZVV5PKKLF22DBGBDAXESUQGFX5BQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.452285782 (25h48m13.491858902s) [noise=true , address=77.90.31.14:4663 , piece_id=3HGB2PS4NQTKCNFBG5CBLHLDJPQRPSZ37HPWV4YFP75KU22PMNKQ, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:37.452516314 (25h48m13.492089434s) [noise=true , address=77.90.31.13:4626 , piece_id=7NQ6UCG27KACEDWPNAONERF47BQTDX5QF7ATL4QEX7ZIPGVSAVXA, seg_pos={0 0}, upload=22709]: Uploading piece... Sep 7 15:16:39.209584838 (25h48m15.249157958s) [upload=22709]: Next returned nil segment Sep 7 15:16:39.209601631 (25h48m15.249174751s) [upload=22709]: Waiting for error group managing segments... Sep 7 15:16:39.645102354 (25h48m15.684675474s) [noise=true , address=45.91.116.11:4119 , piece_id=VWNMGBBFCJPXETKVVU56SZSVZO6OKGEOZDSKAME4P7PYVX7J5O3A, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.650210057 (25h48m15.689783177s) [noise=true , address=45.91.116.13:4153 , piece_id=NLDDITOBVEU6H3HUYUXLM6L5CWR3DUWTUNOPLVJIEKQBNUTMZ5CA, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.664060150 (25h48m15.70363326s) [noise=true , address=45.91.116.10:4081 , piece_id=47OA7ZWX46W2BB5TT5WYZIY6BUYPD72IMVK2O3BNRYU2NCTOLBGA, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.665130097 (25h48m15.704703217s) [noise=true , address=77.90.31.13:4626 , piece_id=7NQ6UCG27KACEDWPNAONERF47BQTDX5QF7ATL4QEX7ZIPGVSAVXA, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.665736809 (25h48m15.705309929s) [noise=true , address=45.91.116.6:4973 , piece_id=IZ3ENT4QFABYZ57LFPVD5CYAV56WQTUBRSDUKVXTHRKBO4DNDJMQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.669793902 (25h48m15.709367022s) [noise=true , address=45.91.116.6:4975 , piece_id=N7HWJUH7YWTW3XNSWFTOCS2WH6LF2PXV2QVNOONVLGTKOJY6YN3A, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.685929598 (25h48m15.725502718s) [noise=true , address=77.90.31.12:4589 , piece_id=L3X22HGVNCLIHIQLLETIYTLWEHZ6B5AQEPAKAQVJFTNQHUMQM5SQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.687531002 (25h48m15.727104122s) [noise=true , address=77.90.31.16:4733 , piece_id=NB2PS6XLXXOWFSID4DXLXHAF26NFDR3G4TSJL5XCIRF5VAASNF4A, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.741425211 (25h48m15.780998331s) [noise=true , address=77.90.31.21:4877 , piece_id=LECKDTF6AQRU7V2RH3FE5M4KHFHTGKSDYJMIWUHE6PMVA3HN43MQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.749466489 (25h48m15.789039609s) [noise=true , address=77.90.31.12:4592 , piece_id=YCTZJCCSJF25EHDO4P2IE2Z3HHFEOGRFLTOJKW7WUMFBTDMKWPQQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.807397822 (25h48m15.846970942s) [noise=true , address=45.91.116.5:4298 , piece_id=DYQ2KJHRRCIUNGX445B6VJMM4VYBUWR5AXAQZ5NYZ34L7QHUGPUQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.815668978 (25h48m15.855242098s) [noise=true , address=77.90.31.15:4699 , piece_id=4J5X5BKJZ5EBCVOOBIHGOYGY72QQBJ4J7SKMVHNT4NXLWASSV23A, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.854077455 (25h48m15.893650575s) [noise=true , address=45.91.116.10:4083 , piece_id=OMUTFXOXUGZ6E57CMXXL6Y2SZVV5PKKLF22DBGBDAXESUQGFX5BQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: <nil> Sep 7 15:16:39.854106962 (25h48m15.893680082s) [seg_pos={0 0}, upload=22709]: Segment reached optimal threshold of 13 pieces. Sep 7 15:16:39.854276235 (25h48m15.893849355s) [noise=true , address=77.90.31.10:4520 , piece_id=TM7MPCW7DYBNOAPNPZA2B3WX3CKNBAYN2MYSWW7VK7WC7SGZI73Q, seg_pos={0 0}, upload=22709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Sep 7 15:16:39.854361609 (25h48m15.893934729s) [noise=true , address=77.90.31.10:4517 , piece_id=WUSH4DCQXP6P62QHMGSY3BZXXI2GBYCDFXIU7LA2BGT4ICVSJFRQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Sep 7 15:16:39.854387168 (25h48m15.893960288s) [noise=true , address=77.90.31.14:4663 , piece_id=3HGB2PS4NQTKCNFBG5CBLHLDJPQRPSZ37HPWV4YFP75KU22PMNKQ, seg_pos={0 0}, upload=22709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:35442->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:35442->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35442->77.90.31.14:4663: use of closed network connection Sep 7 15:16:39.854404782 (25h48m15.893977902s) [seg_pos={0 0}, upload=22709]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:16:39.854505804 (25h48m15.894078924s) [upload=22709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:16:39.854510523 (25h48m15.894083643s) [upload=22709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:16:40.295498076 (25h48m16.335071196s) [upload=22709]: Done uploading segments... Sep 7 15:16:40.295521171 (25h48m16.335094291s) [upload=22709]: Upload finished. err: <nil> Duration: 30.001706712s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [.....XXXX.X..XX..X.......XX.X.XXX......X....X..X........XX.X] duration: 30.001689277s FAIL (2025-09-07 15:15:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:15:24.782282863 (25h47m0.821856013s) [upload=22691]: Starting upload Sep 7 15:15:24.782479720 (25h47m0.82205284s) [upload=22691]: Done starting upload Sep 7 15:15:24.782580914 (25h47m0.822154034s) [upload=22691]: Uploading segments... Sep 7 15:15:24.782799081 (25h47m0.822372201s) [upload=22691]: Got next segment. Inline: false Sep 7 15:15:24.782855409 (25h47m0.822428529s) [upload=22691]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:15:24.782861941 (25h47m0.822435061s) [upload=22691]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:15:25.059881757 (25h47m1.099454877s) [seg_pos={0 0}, upload=22691]: Begin upload segment... Sep 7 15:15:25.059925540 (25h47m1.09949866s) [seg_pos={0 0}, upload=22691]: Done begin upload segment. Sep 7 15:15:25.059946401 (25h47m1.099519521s) [noise=true , address=45.91.116.6:4973 , piece_id=IHQEGLLPT624YC7YMOVUOH5K2B5GJPI5QLU3HDZVIXQMX4K7646A, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:25.060366755 (25h47m1.099939915s) [noise=true , address=45.91.116.14:4189 , piece_id=RSJ47RC42XVMZMNWL7ONFKJPZ4J7RP5WAN4EVD4MPCM7CP6SFYHQ, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:25.060756080 (25h47m1.10032919s) [noise=true , address=45.91.116.8:4047 , piece_id=FO3MF5SEIAUJK3L3D77QPWU7DBZBBNCJH332EZT7VGCAQZGTPN3A, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:25.061045644 (25h47m1.100618824s) [noise=true , address=45.91.116.14:4190 , piece_id=R3WZZGPCD5Z7YZZCMBNIOUDEIQV3GKDRGAI276G6TIW5NM33GTGQ, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:25.061321051 (25h47m1.100894161s) [noise=true , address=45.91.116.8:4048 , piece_id=HDIIFJCMDMCLVTZYJJ2K543MFZWUS23PGP3VL6HNIIY5GLNB42RQ, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:25.061575077 (25h47m1.101148197s) [noise=true , address=45.91.116.10:4082 , piece_id=VRJVJUGAUFUGCI34DFUQKEZKT55PWQU5X66RPX6QRLX6ZYAYF32A, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:25.061789717 (25h47m1.101362837s) [noise=true , address=45.91.116.9:4262 , piece_id=5OJAAUSIOQ4X3B63NNMMLYMAPW2V3ML2HEWDHV3F6HYIZ7JWSIOQ, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:25.061859161 (25h47m1.101432281s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEXALJ3QC5Q2NYYVJAL4YHLGRU6M4ZLIC5XR6ICOLPDFQFMXMOA, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:25.062095994 (25h47m1.101669114s) [noise=true , address=77.90.31.16:4734 , piece_id=C5FS3AT3PSLDOM5ZDZNCWBVUZTLNJ3WQ2KCJDVPCUKHQBPZR36LQ, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:25.062409534 (25h47m1.101982654s) [noise=true , address=45.91.116.12:4227 , piece_id=AGSWXIWCHBXME5ZT5MW7KLBDFQA5TSVPURLZEUTSAZ5IVSSH7HUQ, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:25.062626319 (25h47m1.102199439s) [noise=true , address=77.90.31.12:4590 , piece_id=OLLH7DERH3DESP23PCXLX4VYI4TU5XFYGXLMCUTCYSZXRVIXF3AA, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:25.062844556 (25h47m1.102417676s) [noise=true , address=45.91.116.9:4263 , piece_id=OY22N4D6FLLX4QDJHAVMXBU7IKEQWRNLCJTZ6CPM2SJ6XIG3ORKA, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:25.063106247 (25h47m1.102679367s) [noise=true , address=45.91.116.6:4974 , piece_id=5ZHJU6NEUANRB2VCVY4ZOKK4ZXCGCXS7N6TMURH666ECQRRBJF4A, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:25.063322982 (25h47m1.102896102s) [noise=true , address=77.90.31.15:4699 , piece_id=PPPMLSUNG72TKDJOLPDPN6Z22XCKD54UX3ET5GH6PIVTFFAJO4VQ, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:25.063701756 (25h47m1.103274876s) [noise=true , address=45.91.116.4:4939 , piece_id=FTN2MSF5YCYFDFGM5YRP2V7QSVRI6SNCLABWYRLPUUQXCIYF2JOA, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:25.063955352 (25h47m1.103528472s) [noise=true , address=77.90.31.14:4662 , piece_id=WKL3K73ARSXSCCJD7PZBZRGTL2HWAFAA7QVPURMVMDBW7ATDPDDQ, seg_pos={0 0}, upload=22691]: Uploading piece... Sep 7 15:15:26.943412610 (25h47m2.98298573s) [upload=22691]: Next returned nil segment Sep 7 15:15:26.943426738 (25h47m2.982999858s) [upload=22691]: Waiting for error group managing segments... Sep 7 15:15:27.239666051 (25h47m3.279239171s) [noise=true , address=45.91.116.14:4190 , piece_id=R3WZZGPCD5Z7YZZCMBNIOUDEIQV3GKDRGAI276G6TIW5NM33GTGQ, seg_pos={0 0}, upload=22691]: Done uploading piece. err: <nil> Sep 7 15:15:27.300779784 (25h47m3.340352904s) [noise=true , address=45.91.116.6:4973 , piece_id=IHQEGLLPT624YC7YMOVUOH5K2B5GJPI5QLU3HDZVIXQMX4K7646A, seg_pos={0 0}, upload=22691]: Done uploading piece. err: <nil> Sep 7 15:15:27.303998452 (25h47m3.343571572s) [noise=true , address=45.91.116.9:4262 , piece_id=5OJAAUSIOQ4X3B63NNMMLYMAPW2V3ML2HEWDHV3F6HYIZ7JWSIOQ, seg_pos={0 0}, upload=22691]: Done uploading piece. err: <nil> Sep 7 15:15:27.313598954 (25h47m3.353172074s) [noise=true , address=77.90.31.12:4590 , piece_id=OLLH7DERH3DESP23PCXLX4VYI4TU5XFYGXLMCUTCYSZXRVIXF3AA, seg_pos={0 0}, upload=22691]: Done uploading piece. err: <nil> Sep 7 15:15:27.327102403 (25h47m3.366675523s) [noise=true , address=45.91.116.6:4974 , piece_id=5ZHJU6NEUANRB2VCVY4ZOKK4ZXCGCXS7N6TMURH666ECQRRBJF4A, seg_pos={0 0}, upload=22691]: Done uploading piece. err: <nil> Sep 7 15:15:27.334601483 (25h47m3.374174603s) [noise=true , address=77.90.31.16:4734 , piece_id=C5FS3AT3PSLDOM5ZDZNCWBVUZTLNJ3WQ2KCJDVPCUKHQBPZR36LQ, seg_pos={0 0}, upload=22691]: Done uploading piece. err: <nil> Sep 7 15:15:27.335068186 (25h47m3.374641306s) [noise=true , address=45.91.116.8:4048 , piece_id=HDIIFJCMDMCLVTZYJJ2K543MFZWUS23PGP3VL6HNIIY5GLNB42RQ, seg_pos={0 0}, upload=22691]: Done uploading piece. err: <nil> Sep 7 15:15:27.335163107 (25h47m3.374736227s) [noise=true , address=45.91.116.14:4189 , piece_id=RSJ47RC42XVMZMNWL7ONFKJPZ4J7RP5WAN4EVD4MPCM7CP6SFYHQ, seg_pos={0 0}, upload=22691]: Done uploading piece. err: <nil> Sep 7 15:15:27.351310596 (25h47m3.390883716s) [noise=true , address=45.91.116.12:4227 , piece_id=AGSWXIWCHBXME5ZT5MW7KLBDFQA5TSVPURLZEUTSAZ5IVSSH7HUQ, seg_pos={0 0}, upload=22691]: Done uploading piece. err: <nil> Sep 7 15:15:27.351544694 (25h47m3.391117814s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEXALJ3QC5Q2NYYVJAL4YHLGRU6M4ZLIC5XR6ICOLPDFQFMXMOA, seg_pos={0 0}, upload=22691]: Done uploading piece. err: <nil> Sep 7 15:15:27.360008680 (25h47m3.399581811s) [noise=true , address=45.91.116.8:4047 , piece_id=FO3MF5SEIAUJK3L3D77QPWU7DBZBBNCJH332EZT7VGCAQZGTPN3A, seg_pos={0 0}, upload=22691]: Done uploading piece. err: <nil> Sep 7 15:15:27.380628887 (25h47m3.420202007s) [noise=true , address=45.91.116.9:4263 , piece_id=OY22N4D6FLLX4QDJHAVMXBU7IKEQWRNLCJTZ6CPM2SJ6XIG3ORKA, seg_pos={0 0}, upload=22691]: Done uploading piece. err: <nil> Sep 7 15:15:27.451405731 (25h47m3.490978851s) [noise=true , address=77.90.31.14:4662 , piece_id=WKL3K73ARSXSCCJD7PZBZRGTL2HWAFAA7QVPURMVMDBW7ATDPDDQ, seg_pos={0 0}, upload=22691]: Done uploading piece. err: <nil> Sep 7 15:15:27.451428926 (25h47m3.491002036s) [seg_pos={0 0}, upload=22691]: Segment reached optimal threshold of 13 pieces. Sep 7 15:15:27.451530319 (25h47m3.491103439s) [noise=true , address=77.90.31.15:4699 , piece_id=PPPMLSUNG72TKDJOLPDPN6Z22XCKD54UX3ET5GH6PIVTFFAJO4VQ, seg_pos={0 0}, upload=22691]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Sep 7 15:15:27.451587158 (25h47m3.491160278s) [noise=true , address=45.91.116.10:4082 , piece_id=VRJVJUGAUFUGCI34DFUQKEZKT55PWQU5X66RPX6QRLX6ZYAYF32A, seg_pos={0 0}, upload=22691]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Sep 7 15:15:27.451877123 (25h47m3.491450243s) [noise=true , address=45.91.116.4:4939 , piece_id=FTN2MSF5YCYFDFGM5YRP2V7QSVRI6SNCLABWYRLPUUQXCIYF2JOA, seg_pos={0 0}, upload=22691]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: context canceled; context canceled Sep 7 15:15:27.451914274 (25h47m3.491487384s) [seg_pos={0 0}, upload=22691]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:15:27.451992103 (25h47m3.491565223s) [upload=22691]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:15:27.451996171 (25h47m3.491569281s) [upload=22691]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:15:27.801983272 (25h47m3.841556392s) [upload=22691]: Done uploading segments... Sep 7 15:15:27.802001386 (25h47m3.841574506s) [upload=22691]: Upload finished. err: <nil> Duration: 30.001718624s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [X.....XXXX.X..XX..X.......XX.X.XXX......X....X..X........XX.] duration: 10.049177357s FAIL (2025-09-07 15:15:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:13:43.578634233 (25h45m19.618207403s) [upload=22667]: Starting upload Sep 7 15:13:43.578796113 (25h45m19.618369233s) [upload=22667]: Done starting upload Sep 7 15:13:43.578918979 (25h45m19.618492159s) [upload=22667]: Uploading segments... Sep 7 15:13:43.579291251 (25h45m19.618864401s) [upload=22667]: Got next segment. Inline: false Sep 7 15:13:43.579351677 (25h45m19.618924837s) [upload=22667]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:13:43.579360503 (25h45m19.618933654s) [upload=22667]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:13:44.089965321 (25h45m20.129538441s) [seg_pos={0 0}, upload=22667]: Begin upload segment... Sep 7 15:13:44.090021429 (25h45m20.129594539s) [seg_pos={0 0}, upload=22667]: Done begin upload segment. Sep 7 15:13:44.090047038 (25h45m20.129620158s) [noise=true , address=45.91.116.8:4048 , piece_id=XUHKLIZK6SOTOP4PBFIPBSFZT4UGFCZ5UFZS6HZL2ZXIXGRSIBWA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.090122011 (25h45m20.129695131s) [noise=true , address=77.90.31.10:4520 , piece_id=AAQXM2PY2GCQGJC6NF2LDIWCPWOIX5XSFJAOD35C3PNUELYCKPLA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.090387900 (25h45m20.12996102s) [noise=true , address=77.90.31.14:4663 , piece_id=2QDRS6AQ43BX6YOY3G656UW7GJ6MD2EZBUE733CALTQ472CY66FA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.090642386 (25h45m20.130215506s) [noise=true , address=45.91.116.11:4119 , piece_id=ZRS4AHPXKEH2CDTL2LGFH6W7A6TXOXLAQ4CTM4COXCSLFZCNVM2Q, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.090891824 (25h45m20.130464944s) [noise=true , address=77.90.31.15:4698 , piece_id=AVUXSA5VZDS2LLLHPXQ3ZPEBRSYQZ5UDDQ6737JMRTW7LBW6SSDA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.090905159 (25h45m20.13047828s) [noise=true , address=77.90.31.12:4592 , piece_id=3BJAZIWCO5B3F6KX7GII63QMFDKBL7WMAWUK7J6Y4UKDCXWVC5SA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.091230783 (25h45m20.130803903s) [noise=true , address=77.90.31.10:4518 , piece_id=KHZNMZ5BZOQR727ZS6MNIZHBRZ7QO3GPYVB6RWC4G5K72UKNXQYA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.091430625 (25h45m20.131003745s) [noise=true , address=45.91.116.9:4263 , piece_id=FFWGBED2ZA5LC6WJZYTGGFBFJVVINFXW4ZY2B4IDSWPANSVMZGFQ, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.091493255 (25h45m20.131066375s) [noise=true , address=45.91.116.4:4939 , piece_id=B6LFF54UU674XKLMIIKSZNWYJGEJZWMSVKO3SLTYJI4SKXVWQTRA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.091731601 (25h45m20.131304721s) [noise=true , address=45.91.116.6:4973 , piece_id=HQE7DYI7YWXQR6CAZGE3UNRSESVHDZUO3KY7XI5GM32RLVRXUNYQ, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.091974626 (25h45m20.131547746s) [noise=true , address=45.91.116.7:4011 , piece_id=CODEOP32BZZ3AGSOEQQ7XVHXCYLCK552FHSLNLH65OLP7LV6RK4A, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.092006156 (25h45m20.131579276s) [noise=true , address=45.91.116.13:4153 , piece_id=D4KJSOKPZYMDROAB7DHQIBHFWYZ3MDWXQARYSQJAWP7RAWN7NJ3Q, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.092304276 (25h45m20.131877397s) [noise=true , address=45.91.116.7:4009 , piece_id=L3ULRM4RAOFROTML3Z56ETTUW4BUCEO5DCVQK5BOVTUQIYXL52NA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.092458061 (25h45m20.132031171s) [noise=true , address=45.91.116.6:4975 , piece_id=BBEJBOK5Q67BHPEIP2QWHAQCKRW36XTJRF56FXLXESGJSRSIEW2Q, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.092975351 (25h45m20.132548471s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZNCKC44V37YNGTT2JPPCMBVUHRKLA35AKN4VBEXKC3IQK2SBJZQ, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.093466571 (25h45m20.133039701s) [noise=true , address=45.91.116.4:4937 , piece_id=3NEMSHYLLTIXPKZWBOMM6VGGPR4N2E5E5HWBBXFEB5HBZ3EVSGAQ, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:45.953526323 (25h45m21.993099443s) [upload=22667]: Next returned nil segment Sep 7 15:13:45.953539497 (25h45m21.993112607s) [upload=22667]: Waiting for error group managing segments... Sep 7 15:13:46.295550956 (25h45m22.335124076s) [noise=true , address=77.90.31.10:4520 , piece_id=AAQXM2PY2GCQGJC6NF2LDIWCPWOIX5XSFJAOD35C3PNUELYCKPLA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.297416235 (25h45m22.336989355s) [noise=true , address=45.91.116.11:4119 , piece_id=ZRS4AHPXKEH2CDTL2LGFH6W7A6TXOXLAQ4CTM4COXCSLFZCNVM2Q, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.301612786 (25h45m22.341185906s) [noise=true , address=77.90.31.14:4663 , piece_id=2QDRS6AQ43BX6YOY3G656UW7GJ6MD2EZBUE733CALTQ472CY66FA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.303105583 (25h45m22.342678703s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZNCKC44V37YNGTT2JPPCMBVUHRKLA35AKN4VBEXKC3IQK2SBJZQ, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.323593868 (25h45m22.363166978s) [noise=true , address=45.91.116.4:4937 , piece_id=3NEMSHYLLTIXPKZWBOMM6VGGPR4N2E5E5HWBBXFEB5HBZ3EVSGAQ, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.325887847 (25h45m22.365460968s) [noise=true , address=45.91.116.13:4153 , piece_id=D4KJSOKPZYMDROAB7DHQIBHFWYZ3MDWXQARYSQJAWP7RAWN7NJ3Q, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.345039815 (25h45m22.384612935s) [noise=true , address=45.91.116.7:4011 , piece_id=CODEOP32BZZ3AGSOEQQ7XVHXCYLCK552FHSLNLH65OLP7LV6RK4A, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.346234712 (25h45m22.385807832s) [noise=true , address=45.91.116.8:4048 , piece_id=XUHKLIZK6SOTOP4PBFIPBSFZT4UGFCZ5UFZS6HZL2ZXIXGRSIBWA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.360963456 (25h45m22.400536576s) [noise=true , address=45.91.116.9:4263 , piece_id=FFWGBED2ZA5LC6WJZYTGGFBFJVVINFXW4ZY2B4IDSWPANSVMZGFQ, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.367946438 (25h45m22.407519558s) [noise=true , address=45.91.116.6:4973 , piece_id=HQE7DYI7YWXQR6CAZGE3UNRSESVHDZUO3KY7XI5GM32RLVRXUNYQ, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.369706846 (25h45m22.409279966s) [noise=true , address=77.90.31.12:4592 , piece_id=3BJAZIWCO5B3F6KX7GII63QMFDKBL7WMAWUK7J6Y4UKDCXWVC5SA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.375320077 (25h45m22.414893197s) [noise=true , address=77.90.31.15:4698 , piece_id=AVUXSA5VZDS2LLLHPXQ3ZPEBRSYQZ5UDDQ6737JMRTW7LBW6SSDA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.416577274 (25h45m22.456150394s) [noise=true , address=45.91.116.4:4939 , piece_id=B6LFF54UU674XKLMIIKSZNWYJGEJZWMSVKO3SLTYJI4SKXVWQTRA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.416604717 (25h45m22.456177827s) [seg_pos={0 0}, upload=22667]: Segment reached optimal threshold of 13 pieces. Sep 7 15:13:46.416697093 (25h45m22.456270203s) [noise=true , address=45.91.116.6:4975 , piece_id=BBEJBOK5Q67BHPEIP2QWHAQCKRW36XTJRF56FXLXESGJSRSIEW2Q, seg_pos={0 0}, upload=22667]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Sep 7 15:13:46.416755906 (25h45m22.456329026s) [noise=true , address=45.91.116.7:4009 , piece_id=L3ULRM4RAOFROTML3Z56ETTUW4BUCEO5DCVQK5BOVTUQIYXL52NA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 15:13:46.416844426 (25h45m22.456417546s) [noise=true , address=77.90.31.10:4518 , piece_id=KHZNMZ5BZOQR727ZS6MNIZHBRZ7QO3GPYVB6RWC4G5K72UKNXQYA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3): protocol: expected piece hash; context canceled; EOF Sep 7 15:13:46.416886385 (25h45m22.456459506s) [seg_pos={0 0}, upload=22667]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:13:46.416964305 (25h45m22.456537425s) [upload=22667]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:13:46.416968563 (25h45m22.456541683s) [upload=22667]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:13:46.794794654 (25h45m22.834367764s) [upload=22667]: Done uploading segments... Sep 7 15:13:46.794811156 (25h45m22.834384276s) [upload=22667]: Upload finished. err: <nil> Duration: 10.049206092s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [.X.....XXXX.X..XX..X.......XX.X.XXX......X....X..X........XX] duration: 30.001827443s FAIL (2025-09-07 15:14:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:13:43.578634233 (25h45m19.618207403s) [upload=22667]: Starting upload Sep 7 15:13:43.578796113 (25h45m19.618369233s) [upload=22667]: Done starting upload Sep 7 15:13:43.578918979 (25h45m19.618492159s) [upload=22667]: Uploading segments... Sep 7 15:13:43.579291251 (25h45m19.618864401s) [upload=22667]: Got next segment. Inline: false Sep 7 15:13:43.579351677 (25h45m19.618924837s) [upload=22667]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:13:43.579360503 (25h45m19.618933654s) [upload=22667]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:13:44.089965321 (25h45m20.129538441s) [seg_pos={0 0}, upload=22667]: Begin upload segment... Sep 7 15:13:44.090021429 (25h45m20.129594539s) [seg_pos={0 0}, upload=22667]: Done begin upload segment. Sep 7 15:13:44.090047038 (25h45m20.129620158s) [noise=true , address=45.91.116.8:4048 , piece_id=XUHKLIZK6SOTOP4PBFIPBSFZT4UGFCZ5UFZS6HZL2ZXIXGRSIBWA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.090122011 (25h45m20.129695131s) [noise=true , address=77.90.31.10:4520 , piece_id=AAQXM2PY2GCQGJC6NF2LDIWCPWOIX5XSFJAOD35C3PNUELYCKPLA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.090387900 (25h45m20.12996102s) [noise=true , address=77.90.31.14:4663 , piece_id=2QDRS6AQ43BX6YOY3G656UW7GJ6MD2EZBUE733CALTQ472CY66FA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.090642386 (25h45m20.130215506s) [noise=true , address=45.91.116.11:4119 , piece_id=ZRS4AHPXKEH2CDTL2LGFH6W7A6TXOXLAQ4CTM4COXCSLFZCNVM2Q, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.090891824 (25h45m20.130464944s) [noise=true , address=77.90.31.15:4698 , piece_id=AVUXSA5VZDS2LLLHPXQ3ZPEBRSYQZ5UDDQ6737JMRTW7LBW6SSDA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.090905159 (25h45m20.13047828s) [noise=true , address=77.90.31.12:4592 , piece_id=3BJAZIWCO5B3F6KX7GII63QMFDKBL7WMAWUK7J6Y4UKDCXWVC5SA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.091230783 (25h45m20.130803903s) [noise=true , address=77.90.31.10:4518 , piece_id=KHZNMZ5BZOQR727ZS6MNIZHBRZ7QO3GPYVB6RWC4G5K72UKNXQYA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.091430625 (25h45m20.131003745s) [noise=true , address=45.91.116.9:4263 , piece_id=FFWGBED2ZA5LC6WJZYTGGFBFJVVINFXW4ZY2B4IDSWPANSVMZGFQ, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.091493255 (25h45m20.131066375s) [noise=true , address=45.91.116.4:4939 , piece_id=B6LFF54UU674XKLMIIKSZNWYJGEJZWMSVKO3SLTYJI4SKXVWQTRA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.091731601 (25h45m20.131304721s) [noise=true , address=45.91.116.6:4973 , piece_id=HQE7DYI7YWXQR6CAZGE3UNRSESVHDZUO3KY7XI5GM32RLVRXUNYQ, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.091974626 (25h45m20.131547746s) [noise=true , address=45.91.116.7:4011 , piece_id=CODEOP32BZZ3AGSOEQQ7XVHXCYLCK552FHSLNLH65OLP7LV6RK4A, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.092006156 (25h45m20.131579276s) [noise=true , address=45.91.116.13:4153 , piece_id=D4KJSOKPZYMDROAB7DHQIBHFWYZ3MDWXQARYSQJAWP7RAWN7NJ3Q, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.092304276 (25h45m20.131877397s) [noise=true , address=45.91.116.7:4009 , piece_id=L3ULRM4RAOFROTML3Z56ETTUW4BUCEO5DCVQK5BOVTUQIYXL52NA, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.092458061 (25h45m20.132031171s) [noise=true , address=45.91.116.6:4975 , piece_id=BBEJBOK5Q67BHPEIP2QWHAQCKRW36XTJRF56FXLXESGJSRSIEW2Q, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.092975351 (25h45m20.132548471s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZNCKC44V37YNGTT2JPPCMBVUHRKLA35AKN4VBEXKC3IQK2SBJZQ, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:44.093466571 (25h45m20.133039701s) [noise=true , address=45.91.116.4:4937 , piece_id=3NEMSHYLLTIXPKZWBOMM6VGGPR4N2E5E5HWBBXFEB5HBZ3EVSGAQ, seg_pos={0 0}, upload=22667]: Uploading piece... Sep 7 15:13:45.953526323 (25h45m21.993099443s) [upload=22667]: Next returned nil segment Sep 7 15:13:45.953539497 (25h45m21.993112607s) [upload=22667]: Waiting for error group managing segments... Sep 7 15:13:46.295550956 (25h45m22.335124076s) [noise=true , address=77.90.31.10:4520 , piece_id=AAQXM2PY2GCQGJC6NF2LDIWCPWOIX5XSFJAOD35C3PNUELYCKPLA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.297416235 (25h45m22.336989355s) [noise=true , address=45.91.116.11:4119 , piece_id=ZRS4AHPXKEH2CDTL2LGFH6W7A6TXOXLAQ4CTM4COXCSLFZCNVM2Q, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.301612786 (25h45m22.341185906s) [noise=true , address=77.90.31.14:4663 , piece_id=2QDRS6AQ43BX6YOY3G656UW7GJ6MD2EZBUE733CALTQ472CY66FA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.303105583 (25h45m22.342678703s) [noise=true , address=77.90.31.21:4877 , piece_id=3ZNCKC44V37YNGTT2JPPCMBVUHRKLA35AKN4VBEXKC3IQK2SBJZQ, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.323593868 (25h45m22.363166978s) [noise=true , address=45.91.116.4:4937 , piece_id=3NEMSHYLLTIXPKZWBOMM6VGGPR4N2E5E5HWBBXFEB5HBZ3EVSGAQ, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.325887847 (25h45m22.365460968s) [noise=true , address=45.91.116.13:4153 , piece_id=D4KJSOKPZYMDROAB7DHQIBHFWYZ3MDWXQARYSQJAWP7RAWN7NJ3Q, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.345039815 (25h45m22.384612935s) [noise=true , address=45.91.116.7:4011 , piece_id=CODEOP32BZZ3AGSOEQQ7XVHXCYLCK552FHSLNLH65OLP7LV6RK4A, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.346234712 (25h45m22.385807832s) [noise=true , address=45.91.116.8:4048 , piece_id=XUHKLIZK6SOTOP4PBFIPBSFZT4UGFCZ5UFZS6HZL2ZXIXGRSIBWA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.360963456 (25h45m22.400536576s) [noise=true , address=45.91.116.9:4263 , piece_id=FFWGBED2ZA5LC6WJZYTGGFBFJVVINFXW4ZY2B4IDSWPANSVMZGFQ, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.367946438 (25h45m22.407519558s) [noise=true , address=45.91.116.6:4973 , piece_id=HQE7DYI7YWXQR6CAZGE3UNRSESVHDZUO3KY7XI5GM32RLVRXUNYQ, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.369706846 (25h45m22.409279966s) [noise=true , address=77.90.31.12:4592 , piece_id=3BJAZIWCO5B3F6KX7GII63QMFDKBL7WMAWUK7J6Y4UKDCXWVC5SA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.375320077 (25h45m22.414893197s) [noise=true , address=77.90.31.15:4698 , piece_id=AVUXSA5VZDS2LLLHPXQ3ZPEBRSYQZ5UDDQ6737JMRTW7LBW6SSDA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.416577274 (25h45m22.456150394s) [noise=true , address=45.91.116.4:4939 , piece_id=B6LFF54UU674XKLMIIKSZNWYJGEJZWMSVKO3SLTYJI4SKXVWQTRA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: <nil> Sep 7 15:13:46.416604717 (25h45m22.456177827s) [seg_pos={0 0}, upload=22667]: Segment reached optimal threshold of 13 pieces. Sep 7 15:13:46.416697093 (25h45m22.456270203s) [noise=true , address=45.91.116.6:4975 , piece_id=BBEJBOK5Q67BHPEIP2QWHAQCKRW36XTJRF56FXLXESGJSRSIEW2Q, seg_pos={0 0}, upload=22667]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Sep 7 15:13:46.416755906 (25h45m22.456329026s) [noise=true , address=45.91.116.7:4009 , piece_id=L3ULRM4RAOFROTML3Z56ETTUW4BUCEO5DCVQK5BOVTUQIYXL52NA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 7 15:13:46.416844426 (25h45m22.456417546s) [noise=true , address=77.90.31.10:4518 , piece_id=KHZNMZ5BZOQR727ZS6MNIZHBRZ7QO3GPYVB6RWC4G5K72UKNXQYA, seg_pos={0 0}, upload=22667]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3): protocol: expected piece hash; context canceled; EOF Sep 7 15:13:46.416886385 (25h45m22.456459506s) [seg_pos={0 0}, upload=22667]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:13:46.416964305 (25h45m22.456537425s) [upload=22667]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:13:46.416968563 (25h45m22.456541683s) [upload=22667]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:13:46.794794654 (25h45m22.834367764s) [upload=22667]: Done uploading segments... Sep 7 15:13:46.794811156 (25h45m22.834384276s) [upload=22667]: Upload finished. err: <nil> Duration: 30.001851028s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [..X.....XXXX.X..XX..X.......XX.X.XXX......X....X..X........X] duration: 30.00133364s FAIL (2025-09-07 15:13:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:13:12.092486145 (25h44m48.132059266s) [upload=22658]: Starting upload Sep 7 15:13:12.092542032 (25h44m48.132115152s) [upload=22658]: Done starting upload Sep 7 15:13:12.092716787 (25h44m48.132289897s) [upload=22658]: Uploading segments... Sep 7 15:13:12.092750221 (25h44m48.132323341s) [upload=22658]: Got next segment. Inline: false Sep 7 15:13:12.092783024 (25h44m48.132356144s) [upload=22658]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:13:12.092787292 (25h44m48.132360412s) [upload=22658]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:13:12.638606844 (25h44m48.678179964s) [seg_pos={0 0}, upload=22658]: Begin upload segment... Sep 7 15:13:12.638659485 (25h44m48.678232595s) [seg_pos={0 0}, upload=22658]: Done begin upload segment. Sep 7 15:13:12.638854037 (25h44m48.678427157s) [noise=true , address=45.91.116.10:4081 , piece_id=WH3L2UAMG43Y3BHZYZNBCF4XFUMJ54PRO6P2YXW2KOS4FACACJ6Q, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:12.639638108 (25h44m48.679211218s) [noise=true , address=77.90.31.18:4805 , piece_id=GAATHP4KU5WTZXGDRYYLPA5HWIHGWTSYQBANMZHZ6EEYLIGL4V4A, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:12.639916500 (25h44m48.67948962s) [noise=true , address=77.90.31.14:4663 , piece_id=Y6PRXRFF7SZNLJHC5JQFT376MLKCQZ2BEUW4BGYWJMEDNNLWKVWA, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:12.640626719 (25h44m48.68019984s) [noise=true , address=45.91.116.8:4047 , piece_id=BICDXNIAMFQ7BS3DW7GYGSBXFFUDWOALG34L6QAEYVNJWIO24RZA, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:12.640926754 (25h44m48.680499874s) [noise=true , address=45.91.116.5:4298 , piece_id=BNFH46A7NIR5EM5VZTERA4ZLW6DX4XCZGJSRUGLW7TJ5IC57CBBA, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:12.641337950 (25h44m48.68091107s) [noise=true , address=77.90.31.12:4589 , piece_id=273IPFZRB2JJD35QT3J77A3LA3J4IMJA7A6BC55P3UD3H33LVXQA, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:12.641787711 (25h44m48.681360831s) [noise=true , address=45.91.116.13:4154 , piece_id=QBXHGMPHME7TCWAJKE7PLLPE2W4NZ7EIEEFX6B4OSDX4JNSCBTEQ, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:12.642727520 (25h44m48.68230064s) [noise=true , address=45.91.116.10:4083 , piece_id=7V3KPLD5SPOIJNVOQW4AICV2JTZIVJXYOUIFQNF4OSBX6WPN2A7Q, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:12.643160989 (25h44m48.682734109s) [noise=true , address=45.91.116.6:4974 , piece_id=B24ISW26QCGFMUJKSDSBIIMTD4FX7H4LEU3KOL2SG2UUH3FQUDBQ, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:12.643587154 (25h44m48.683160274s) [noise=true , address=77.90.31.13:4628 , piece_id=WRLQRSZ23SHUG323VJ5NEZQZFGAYCEWBWOJZXTQYMNV7XAX2T4EQ, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:12.643879454 (25h44m48.683452574s) [noise=true , address=77.90.31.21:4877 , piece_id=YBZKUHIOAQUUPGXAHSTL6FTSXRKF7D2RB37E64DTB6ERO5V4N5FA, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:12.644772553 (25h44m48.684345673s) [noise=true , address=45.91.116.7:4011 , piece_id=FY64PFAQVGFSQOU3TWNP2TI6DZWTAYWDFOZGHPSBCTM35TDQZKWQ, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:12.645113304 (25h44m48.684686425s) [noise=true , address=77.90.31.13:4627 , piece_id=UZFD4RDTGNEBYM4VT54ZPE53YHNFHQAYFPZWBW2IV2QOIFF2LHMA, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:12.645389102 (25h44m48.684962222s) [noise=true , address=45.91.116.6:4973 , piece_id=YHWYX33EIIH5CFPDHUK3QQB6IR2NCNGQY2VJJ4CGNW6FIX2QTVCA, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:12.645736698 (25h44m48.685309818s) [noise=true , address=45.91.116.13:4153 , piece_id=ZUEMO3UQ27RFZRUNAVSIC32AFXJ4PZCJKLMSFWPRLFRHT3APPR6Q, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:12.646111054 (25h44m48.685684174s) [noise=true , address=45.91.116.9:4263 , piece_id=EJO6ICDUO5TSNAU4E4ZYQWELEEOERGD7KHCPSOK633G3EKB3CD7Q, seg_pos={0 0}, upload=22658]: Uploading piece... Sep 7 15:13:14.461964905 (25h44m50.501538025s) [upload=22658]: Next returned nil segment Sep 7 15:13:14.461980214 (25h44m50.501553334s) [upload=22658]: Waiting for error group managing segments... Sep 7 15:13:14.819616362 (25h44m50.859189482s) [noise=true , address=77.90.31.18:4805 , piece_id=GAATHP4KU5WTZXGDRYYLPA5HWIHGWTSYQBANMZHZ6EEYLIGL4V4A, seg_pos={0 0}, upload=22658]: Done uploading piece. err: <nil> Sep 7 15:13:14.847324202 (25h44m50.886897322s) [noise=true , address=77.90.31.13:4627 , piece_id=UZFD4RDTGNEBYM4VT54ZPE53YHNFHQAYFPZWBW2IV2QOIFF2LHMA, seg_pos={0 0}, upload=22658]: Done uploading piece. err: <nil> Sep 7 15:13:14.850263386 (25h44m50.889836506s) [noise=true , address=45.91.116.6:4973 , piece_id=YHWYX33EIIH5CFPDHUK3QQB6IR2NCNGQY2VJJ4CGNW6FIX2QTVCA, seg_pos={0 0}, upload=22658]: Done uploading piece. err: <nil> Sep 7 15:13:14.856277704 (25h44m50.895850824s) [noise=true , address=45.91.116.5:4298 , piece_id=BNFH46A7NIR5EM5VZTERA4ZLW6DX4XCZGJSRUGLW7TJ5IC57CBBA, seg_pos={0 0}, upload=22658]: Done uploading piece. err: <nil> Sep 7 15:13:14.858279005 (25h44m50.897852125s) [noise=true , address=45.91.116.13:4154 , piece_id=QBXHGMPHME7TCWAJKE7PLLPE2W4NZ7EIEEFX6B4OSDX4JNSCBTEQ, seg_pos={0 0}, upload=22658]: Done uploading piece. err: <nil> Sep 7 15:13:14.860162238 (25h44m50.899735358s) [noise=true , address=45.91.116.9:4263 , piece_id=EJO6ICDUO5TSNAU4E4ZYQWELEEOERGD7KHCPSOK633G3EKB3CD7Q, seg_pos={0 0}, upload=22658]: Done uploading piece. err: <nil> Sep 7 15:13:14.860860283 (25h44m50.900433413s) [noise=true , address=45.91.116.10:4081 , piece_id=WH3L2UAMG43Y3BHZYZNBCF4XFUMJ54PRO6P2YXW2KOS4FACACJ6Q, seg_pos={0 0}, upload=22658]: Done uploading piece. err: <nil> Sep 7 15:13:14.862194346 (25h44m50.901767466s) [noise=true , address=77.90.31.12:4589 , piece_id=273IPFZRB2JJD35QT3J77A3LA3J4IMJA7A6BC55P3UD3H33LVXQA, seg_pos={0 0}, upload=22658]: Done uploading piece. err: <nil> Sep 7 15:13:14.867846542 (25h44m50.907419662s) [noise=true , address=45.91.116.8:4047 , piece_id=BICDXNIAMFQ7BS3DW7GYGSBXFFUDWOALG34L6QAEYVNJWIO24RZA, seg_pos={0 0}, upload=22658]: Done uploading piece. err: <nil> Sep 7 15:13:14.869622420 (25h44m50.90919554s) [noise=true , address=45.91.116.10:4083 , piece_id=7V3KPLD5SPOIJNVOQW4AICV2JTZIVJXYOUIFQNF4OSBX6WPN2A7Q, seg_pos={0 0}, upload=22658]: Done uploading piece. err: <nil> Sep 7 15:13:14.881682779 (25h44m50.921255889s) [noise=true , address=45.91.116.13:4153 , piece_id=ZUEMO3UQ27RFZRUNAVSIC32AFXJ4PZCJKLMSFWPRLFRHT3APPR6Q, seg_pos={0 0}, upload=22658]: Done uploading piece. err: <nil> Sep 7 15:13:14.924846152 (25h44m50.964419273s) [noise=true , address=77.90.31.13:4628 , piece_id=WRLQRSZ23SHUG323VJ5NEZQZFGAYCEWBWOJZXTQYMNV7XAX2T4EQ, seg_pos={0 0}, upload=22658]: Done uploading piece. err: <nil> Sep 7 15:13:14.980449040 (25h44m51.02002216s) [noise=true , address=77.90.31.21:4877 , piece_id=YBZKUHIOAQUUPGXAHSTL6FTSXRKF7D2RB37E64DTB6ERO5V4N5FA, seg_pos={0 0}, upload=22658]: Done uploading piece. err: <nil> Sep 7 15:13:14.980471393 (25h44m51.020044513s) [seg_pos={0 0}, upload=22658]: Segment reached optimal threshold of 13 pieces. Sep 7 15:13:14.980576775 (25h44m51.020149895s) [noise=true , address=45.91.116.6:4974 , piece_id=B24ISW26QCGFMUJKSDSBIIMTD4FX7H4LEU3KOL2SG2UUH3FQUDBQ, seg_pos={0 0}, upload=22658]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 7 15:13:14.980794181 (25h44m51.020367301s) [noise=true , address=77.90.31.14:4663 , piece_id=Y6PRXRFF7SZNLJHC5JQFT376MLKCQZ2BEUW4BGYWJMEDNNLWKVWA, seg_pos={0 0}, upload=22658]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:51966->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:51966->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51966->77.90.31.14:4663: use of closed network connection Sep 7 15:13:14.980876739 (25h44m51.020449859s) [noise=true , address=45.91.116.7:4011 , piece_id=FY64PFAQVGFSQOU3TWNP2TI6DZWTAYWDFOZGHPSBCTM35TDQZKWQ, seg_pos={0 0}, upload=22658]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: write tcp 188.166.225.170:53972->45.91.116.7:4011: use of closed network connection; write tcp 188.166.225.170:53972->45.91.116.7:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53972->45.91.116.7:4011: use of closed network connection Sep 7 15:13:14.980889904 (25h44m51.020463024s) [seg_pos={0 0}, upload=22658]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:13:14.980968574 (25h44m51.020541694s) [upload=22658]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:13:14.980975137 (25h44m51.020548257s) [upload=22658]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:13:15.695714774 (25h44m51.735287894s) [upload=22658]: Done uploading segments... Sep 7 15:13:15.695734962 (25h44m51.735308083s) [upload=22658]: Upload finished. err: <nil> Duration: 30.001361954s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [X..X.....XXXX.X..XX..X.......XX.X.XXX......X....X..X........] duration: 8.575790327s FAIL (2025-09-07 15:12:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:06:09.477287454 (25h37m45.516860564s) [upload=22556]: Starting upload Sep 7 15:06:09.477341047 (25h37m45.516914167s) [upload=22556]: Done starting upload Sep 7 15:06:09.477516443 (25h37m45.517089563s) [upload=22556]: Uploading segments... Sep 7 15:06:09.477550999 (25h37m45.517124109s) [upload=22556]: Got next segment. Inline: false Sep 7 15:06:09.477593901 (25h37m45.517167021s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:06:09.477598099 (25h37m45.517171219s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:06:09.763314436 (25h37m45.802887556s) [seg_pos={0 0}, upload=22556]: Begin upload segment... Sep 7 15:06:09.763361535 (25h37m45.802934655s) [seg_pos={0 0}, upload=22556]: Done begin upload segment. Sep 7 15:06:09.763384840 (25h37m45.80295796s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763420399 (25h37m45.802993519s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763812789 (25h37m45.803385919s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763990269 (25h37m45.803563389s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764372309 (25h37m45.80394543s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764475748 (25h37m45.804048868s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764744903 (25h37m45.804318023s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764786313 (25h37m45.804359433s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765207568 (25h37m45.804780688s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765475832 (25h37m45.805048952s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765740819 (25h37m45.805313939s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766083895 (25h37m45.805657015s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766410420 (25h37m45.80598357s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766463101 (25h37m45.806036211s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766743106 (25h37m45.806316226s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.767034234 (25h37m45.806607364s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:11.624806047 (25h37m47.664379167s) [upload=22556]: Next returned nil segment Sep 7 15:06:11.624840563 (25h37m47.664413693s) [upload=22556]: Waiting for error group managing segments... Sep 7 15:06:11.945497440 (25h37m47.98507056s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.948932752 (25h37m47.988505873s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.969614708 (25h37m48.009187818s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.990585857 (25h37m48.030158967s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.998861533 (25h37m48.038434653s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.003987873 (25h37m48.043560993s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.004947169 (25h37m48.044520299s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.010614082 (25h37m48.050187202s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.024736596 (25h37m48.064309716s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.036179973 (25h37m48.075753094s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.056595439 (25h37m48.096168559s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.083593591 (25h37m48.123166761s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094768284 (25h37m48.134341404s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094785115 (25h37m48.134358235s) [seg_pos={0 0}, upload=22556]: Segment reached optimal threshold of 13 pieces. Sep 7 15:06:12.094966282 (25h37m48.134539402s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095094788 (25h37m48.134667909s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection Sep 7 15:06:12.095123533 (25h37m48.134696643s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095165353 (25h37m48.134738473s) [seg_pos={0 0}, upload=22556]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:06:12.095253583 (25h37m48.134826703s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:06:12.095259113 (25h37m48.134832223s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:06:12.405199943 (25h37m48.444773134s) [upload=22556]: Done uploading segments... Sep 7 15:06:12.405225172 (25h37m48.444798302s) [upload=22556]: Upload finished. err: <nil> Duration: 8.575816055s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [XX..X.....XXXX.X..XX..X.......XX.X.XXX......X....X..X.......] duration: 17.297919439s FAIL (2025-09-07 15:11:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:06:09.477287454 (25h37m45.516860564s) [upload=22556]: Starting upload Sep 7 15:06:09.477341047 (25h37m45.516914167s) [upload=22556]: Done starting upload Sep 7 15:06:09.477516443 (25h37m45.517089563s) [upload=22556]: Uploading segments... Sep 7 15:06:09.477550999 (25h37m45.517124109s) [upload=22556]: Got next segment. Inline: false Sep 7 15:06:09.477593901 (25h37m45.517167021s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:06:09.477598099 (25h37m45.517171219s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:06:09.763314436 (25h37m45.802887556s) [seg_pos={0 0}, upload=22556]: Begin upload segment... Sep 7 15:06:09.763361535 (25h37m45.802934655s) [seg_pos={0 0}, upload=22556]: Done begin upload segment. Sep 7 15:06:09.763384840 (25h37m45.80295796s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763420399 (25h37m45.802993519s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763812789 (25h37m45.803385919s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763990269 (25h37m45.803563389s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764372309 (25h37m45.80394543s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764475748 (25h37m45.804048868s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764744903 (25h37m45.804318023s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764786313 (25h37m45.804359433s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765207568 (25h37m45.804780688s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765475832 (25h37m45.805048952s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765740819 (25h37m45.805313939s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766083895 (25h37m45.805657015s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766410420 (25h37m45.80598357s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766463101 (25h37m45.806036211s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766743106 (25h37m45.806316226s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.767034234 (25h37m45.806607364s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:11.624806047 (25h37m47.664379167s) [upload=22556]: Next returned nil segment Sep 7 15:06:11.624840563 (25h37m47.664413693s) [upload=22556]: Waiting for error group managing segments... Sep 7 15:06:11.945497440 (25h37m47.98507056s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.948932752 (25h37m47.988505873s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.969614708 (25h37m48.009187818s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.990585857 (25h37m48.030158967s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.998861533 (25h37m48.038434653s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.003987873 (25h37m48.043560993s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.004947169 (25h37m48.044520299s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.010614082 (25h37m48.050187202s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.024736596 (25h37m48.064309716s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.036179973 (25h37m48.075753094s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.056595439 (25h37m48.096168559s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.083593591 (25h37m48.123166761s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094768284 (25h37m48.134341404s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094785115 (25h37m48.134358235s) [seg_pos={0 0}, upload=22556]: Segment reached optimal threshold of 13 pieces. Sep 7 15:06:12.094966282 (25h37m48.134539402s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095094788 (25h37m48.134667909s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection Sep 7 15:06:12.095123533 (25h37m48.134696643s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095165353 (25h37m48.134738473s) [seg_pos={0 0}, upload=22556]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:06:12.095253583 (25h37m48.134826703s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:06:12.095259113 (25h37m48.134832223s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:06:12.405199943 (25h37m48.444773134s) [upload=22556]: Done uploading segments... Sep 7 15:06:12.405225172 (25h37m48.444798302s) [upload=22556]: Upload finished. err: <nil> Duration: 17.297947264s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [.XX..X.....XXXX.X..XX..X.......XX.X.XXX......X....X..X......] duration: 7.477639331s FAIL (2025-09-07 15:10:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:06:09.477287454 (25h37m45.516860564s) [upload=22556]: Starting upload Sep 7 15:06:09.477341047 (25h37m45.516914167s) [upload=22556]: Done starting upload Sep 7 15:06:09.477516443 (25h37m45.517089563s) [upload=22556]: Uploading segments... Sep 7 15:06:09.477550999 (25h37m45.517124109s) [upload=22556]: Got next segment. Inline: false Sep 7 15:06:09.477593901 (25h37m45.517167021s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:06:09.477598099 (25h37m45.517171219s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:06:09.763314436 (25h37m45.802887556s) [seg_pos={0 0}, upload=22556]: Begin upload segment... Sep 7 15:06:09.763361535 (25h37m45.802934655s) [seg_pos={0 0}, upload=22556]: Done begin upload segment. Sep 7 15:06:09.763384840 (25h37m45.80295796s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763420399 (25h37m45.802993519s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763812789 (25h37m45.803385919s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763990269 (25h37m45.803563389s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764372309 (25h37m45.80394543s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764475748 (25h37m45.804048868s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764744903 (25h37m45.804318023s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764786313 (25h37m45.804359433s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765207568 (25h37m45.804780688s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765475832 (25h37m45.805048952s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765740819 (25h37m45.805313939s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766083895 (25h37m45.805657015s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766410420 (25h37m45.80598357s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766463101 (25h37m45.806036211s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766743106 (25h37m45.806316226s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.767034234 (25h37m45.806607364s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:11.624806047 (25h37m47.664379167s) [upload=22556]: Next returned nil segment Sep 7 15:06:11.624840563 (25h37m47.664413693s) [upload=22556]: Waiting for error group managing segments... Sep 7 15:06:11.945497440 (25h37m47.98507056s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.948932752 (25h37m47.988505873s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.969614708 (25h37m48.009187818s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.990585857 (25h37m48.030158967s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.998861533 (25h37m48.038434653s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.003987873 (25h37m48.043560993s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.004947169 (25h37m48.044520299s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.010614082 (25h37m48.050187202s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.024736596 (25h37m48.064309716s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.036179973 (25h37m48.075753094s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.056595439 (25h37m48.096168559s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.083593591 (25h37m48.123166761s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094768284 (25h37m48.134341404s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094785115 (25h37m48.134358235s) [seg_pos={0 0}, upload=22556]: Segment reached optimal threshold of 13 pieces. Sep 7 15:06:12.094966282 (25h37m48.134539402s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095094788 (25h37m48.134667909s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection Sep 7 15:06:12.095123533 (25h37m48.134696643s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095165353 (25h37m48.134738473s) [seg_pos={0 0}, upload=22556]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:06:12.095253583 (25h37m48.134826703s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:06:12.095259113 (25h37m48.134832223s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:06:12.405199943 (25h37m48.444773134s) [upload=22556]: Done uploading segments... Sep 7 15:06:12.405225172 (25h37m48.444798302s) [upload=22556]: Upload finished. err: <nil> Duration: 7.477664099s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [..XX..X.....XXXX.X..XX..X.......XX.X.XXX......X....X..X.....] duration: 15.007502816s FAIL (2025-09-07 15:10:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:06:09.477287454 (25h37m45.516860564s) [upload=22556]: Starting upload Sep 7 15:06:09.477341047 (25h37m45.516914167s) [upload=22556]: Done starting upload Sep 7 15:06:09.477516443 (25h37m45.517089563s) [upload=22556]: Uploading segments... Sep 7 15:06:09.477550999 (25h37m45.517124109s) [upload=22556]: Got next segment. Inline: false Sep 7 15:06:09.477593901 (25h37m45.517167021s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:06:09.477598099 (25h37m45.517171219s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:06:09.763314436 (25h37m45.802887556s) [seg_pos={0 0}, upload=22556]: Begin upload segment... Sep 7 15:06:09.763361535 (25h37m45.802934655s) [seg_pos={0 0}, upload=22556]: Done begin upload segment. Sep 7 15:06:09.763384840 (25h37m45.80295796s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763420399 (25h37m45.802993519s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763812789 (25h37m45.803385919s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763990269 (25h37m45.803563389s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764372309 (25h37m45.80394543s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764475748 (25h37m45.804048868s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764744903 (25h37m45.804318023s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764786313 (25h37m45.804359433s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765207568 (25h37m45.804780688s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765475832 (25h37m45.805048952s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765740819 (25h37m45.805313939s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766083895 (25h37m45.805657015s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766410420 (25h37m45.80598357s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766463101 (25h37m45.806036211s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766743106 (25h37m45.806316226s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.767034234 (25h37m45.806607364s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:11.624806047 (25h37m47.664379167s) [upload=22556]: Next returned nil segment Sep 7 15:06:11.624840563 (25h37m47.664413693s) [upload=22556]: Waiting for error group managing segments... Sep 7 15:06:11.945497440 (25h37m47.98507056s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.948932752 (25h37m47.988505873s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.969614708 (25h37m48.009187818s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.990585857 (25h37m48.030158967s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.998861533 (25h37m48.038434653s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.003987873 (25h37m48.043560993s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.004947169 (25h37m48.044520299s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.010614082 (25h37m48.050187202s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.024736596 (25h37m48.064309716s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.036179973 (25h37m48.075753094s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.056595439 (25h37m48.096168559s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.083593591 (25h37m48.123166761s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094768284 (25h37m48.134341404s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094785115 (25h37m48.134358235s) [seg_pos={0 0}, upload=22556]: Segment reached optimal threshold of 13 pieces. Sep 7 15:06:12.094966282 (25h37m48.134539402s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095094788 (25h37m48.134667909s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection Sep 7 15:06:12.095123533 (25h37m48.134696643s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095165353 (25h37m48.134738473s) [seg_pos={0 0}, upload=22556]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:06:12.095253583 (25h37m48.134826703s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:06:12.095259113 (25h37m48.134832223s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:06:12.405199943 (25h37m48.444773134s) [upload=22556]: Done uploading segments... Sep 7 15:06:12.405225172 (25h37m48.444798302s) [upload=22556]: Upload finished. err: <nil> Duration: 15.007530209s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [...XX..X.....XXXX.X..XX..X.......XX.X.XXX......X....X..X....] duration: 27.746862829s FAIL (2025-09-07 15:09:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:06:09.477287454 (25h37m45.516860564s) [upload=22556]: Starting upload Sep 7 15:06:09.477341047 (25h37m45.516914167s) [upload=22556]: Done starting upload Sep 7 15:06:09.477516443 (25h37m45.517089563s) [upload=22556]: Uploading segments... Sep 7 15:06:09.477550999 (25h37m45.517124109s) [upload=22556]: Got next segment. Inline: false Sep 7 15:06:09.477593901 (25h37m45.517167021s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:06:09.477598099 (25h37m45.517171219s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:06:09.763314436 (25h37m45.802887556s) [seg_pos={0 0}, upload=22556]: Begin upload segment... Sep 7 15:06:09.763361535 (25h37m45.802934655s) [seg_pos={0 0}, upload=22556]: Done begin upload segment. Sep 7 15:06:09.763384840 (25h37m45.80295796s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763420399 (25h37m45.802993519s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763812789 (25h37m45.803385919s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763990269 (25h37m45.803563389s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764372309 (25h37m45.80394543s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764475748 (25h37m45.804048868s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764744903 (25h37m45.804318023s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764786313 (25h37m45.804359433s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765207568 (25h37m45.804780688s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765475832 (25h37m45.805048952s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765740819 (25h37m45.805313939s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766083895 (25h37m45.805657015s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766410420 (25h37m45.80598357s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766463101 (25h37m45.806036211s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766743106 (25h37m45.806316226s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.767034234 (25h37m45.806607364s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:11.624806047 (25h37m47.664379167s) [upload=22556]: Next returned nil segment Sep 7 15:06:11.624840563 (25h37m47.664413693s) [upload=22556]: Waiting for error group managing segments... Sep 7 15:06:11.945497440 (25h37m47.98507056s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.948932752 (25h37m47.988505873s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.969614708 (25h37m48.009187818s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.990585857 (25h37m48.030158967s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.998861533 (25h37m48.038434653s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.003987873 (25h37m48.043560993s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.004947169 (25h37m48.044520299s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.010614082 (25h37m48.050187202s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.024736596 (25h37m48.064309716s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.036179973 (25h37m48.075753094s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.056595439 (25h37m48.096168559s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.083593591 (25h37m48.123166761s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094768284 (25h37m48.134341404s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094785115 (25h37m48.134358235s) [seg_pos={0 0}, upload=22556]: Segment reached optimal threshold of 13 pieces. Sep 7 15:06:12.094966282 (25h37m48.134539402s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095094788 (25h37m48.134667909s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection Sep 7 15:06:12.095123533 (25h37m48.134696643s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095165353 (25h37m48.134738473s) [seg_pos={0 0}, upload=22556]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:06:12.095253583 (25h37m48.134826703s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:06:12.095259113 (25h37m48.134832223s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:06:12.405199943 (25h37m48.444773134s) [upload=22556]: Done uploading segments... Sep 7 15:06:12.405225172 (25h37m48.444798302s) [upload=22556]: Upload finished. err: <nil> Duration: 27.746890693s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [X...XX..X.....XXXX.X..XX..X.......XX.X.XXX......X....X..X...] duration: 10.043113375s FAIL (2025-09-07 15:08:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:06:09.477287454 (25h37m45.516860564s) [upload=22556]: Starting upload Sep 7 15:06:09.477341047 (25h37m45.516914167s) [upload=22556]: Done starting upload Sep 7 15:06:09.477516443 (25h37m45.517089563s) [upload=22556]: Uploading segments... Sep 7 15:06:09.477550999 (25h37m45.517124109s) [upload=22556]: Got next segment. Inline: false Sep 7 15:06:09.477593901 (25h37m45.517167021s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:06:09.477598099 (25h37m45.517171219s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:06:09.763314436 (25h37m45.802887556s) [seg_pos={0 0}, upload=22556]: Begin upload segment... Sep 7 15:06:09.763361535 (25h37m45.802934655s) [seg_pos={0 0}, upload=22556]: Done begin upload segment. Sep 7 15:06:09.763384840 (25h37m45.80295796s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763420399 (25h37m45.802993519s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763812789 (25h37m45.803385919s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763990269 (25h37m45.803563389s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764372309 (25h37m45.80394543s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764475748 (25h37m45.804048868s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764744903 (25h37m45.804318023s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764786313 (25h37m45.804359433s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765207568 (25h37m45.804780688s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765475832 (25h37m45.805048952s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765740819 (25h37m45.805313939s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766083895 (25h37m45.805657015s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766410420 (25h37m45.80598357s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766463101 (25h37m45.806036211s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766743106 (25h37m45.806316226s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.767034234 (25h37m45.806607364s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:11.624806047 (25h37m47.664379167s) [upload=22556]: Next returned nil segment Sep 7 15:06:11.624840563 (25h37m47.664413693s) [upload=22556]: Waiting for error group managing segments... Sep 7 15:06:11.945497440 (25h37m47.98507056s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.948932752 (25h37m47.988505873s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.969614708 (25h37m48.009187818s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.990585857 (25h37m48.030158967s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.998861533 (25h37m48.038434653s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.003987873 (25h37m48.043560993s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.004947169 (25h37m48.044520299s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.010614082 (25h37m48.050187202s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.024736596 (25h37m48.064309716s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.036179973 (25h37m48.075753094s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.056595439 (25h37m48.096168559s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.083593591 (25h37m48.123166761s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094768284 (25h37m48.134341404s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094785115 (25h37m48.134358235s) [seg_pos={0 0}, upload=22556]: Segment reached optimal threshold of 13 pieces. Sep 7 15:06:12.094966282 (25h37m48.134539402s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095094788 (25h37m48.134667909s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection Sep 7 15:06:12.095123533 (25h37m48.134696643s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095165353 (25h37m48.134738473s) [seg_pos={0 0}, upload=22556]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:06:12.095253583 (25h37m48.134826703s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:06:12.095259113 (25h37m48.134832223s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:06:12.405199943 (25h37m48.444773134s) [upload=22556]: Done uploading segments... Sep 7 15:06:12.405225172 (25h37m48.444798302s) [upload=22556]: Upload finished. err: <nil> Duration: 10.043153681s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [.X...XX..X.....XXXX.X..XX..X.......XX.X.XXX......X....X..X..] duration: 16.870168023s FAIL (2025-09-07 15:08:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:06:09.477287454 (25h37m45.516860564s) [upload=22556]: Starting upload Sep 7 15:06:09.477341047 (25h37m45.516914167s) [upload=22556]: Done starting upload Sep 7 15:06:09.477516443 (25h37m45.517089563s) [upload=22556]: Uploading segments... Sep 7 15:06:09.477550999 (25h37m45.517124109s) [upload=22556]: Got next segment. Inline: false Sep 7 15:06:09.477593901 (25h37m45.517167021s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:06:09.477598099 (25h37m45.517171219s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:06:09.763314436 (25h37m45.802887556s) [seg_pos={0 0}, upload=22556]: Begin upload segment... Sep 7 15:06:09.763361535 (25h37m45.802934655s) [seg_pos={0 0}, upload=22556]: Done begin upload segment. Sep 7 15:06:09.763384840 (25h37m45.80295796s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763420399 (25h37m45.802993519s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763812789 (25h37m45.803385919s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763990269 (25h37m45.803563389s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764372309 (25h37m45.80394543s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764475748 (25h37m45.804048868s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764744903 (25h37m45.804318023s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764786313 (25h37m45.804359433s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765207568 (25h37m45.804780688s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765475832 (25h37m45.805048952s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765740819 (25h37m45.805313939s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766083895 (25h37m45.805657015s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766410420 (25h37m45.80598357s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766463101 (25h37m45.806036211s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766743106 (25h37m45.806316226s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.767034234 (25h37m45.806607364s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:11.624806047 (25h37m47.664379167s) [upload=22556]: Next returned nil segment Sep 7 15:06:11.624840563 (25h37m47.664413693s) [upload=22556]: Waiting for error group managing segments... Sep 7 15:06:11.945497440 (25h37m47.98507056s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.948932752 (25h37m47.988505873s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.969614708 (25h37m48.009187818s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.990585857 (25h37m48.030158967s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.998861533 (25h37m48.038434653s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.003987873 (25h37m48.043560993s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.004947169 (25h37m48.044520299s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.010614082 (25h37m48.050187202s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.024736596 (25h37m48.064309716s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.036179973 (25h37m48.075753094s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.056595439 (25h37m48.096168559s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.083593591 (25h37m48.123166761s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094768284 (25h37m48.134341404s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094785115 (25h37m48.134358235s) [seg_pos={0 0}, upload=22556]: Segment reached optimal threshold of 13 pieces. Sep 7 15:06:12.094966282 (25h37m48.134539402s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095094788 (25h37m48.134667909s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection Sep 7 15:06:12.095123533 (25h37m48.134696643s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095165353 (25h37m48.134738473s) [seg_pos={0 0}, upload=22556]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:06:12.095253583 (25h37m48.134826703s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:06:12.095259113 (25h37m48.134832223s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:06:12.405199943 (25h37m48.444773134s) [upload=22556]: Done uploading segments... Sep 7 15:06:12.405225172 (25h37m48.444798302s) [upload=22556]: Upload finished. err: <nil> Duration: 16.87019233s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [X.X...XX..X.....XXXX.X..XX..X.......XX.X.XXX......X....X..X.] duration: 13.382723969s FAIL (2025-09-07 15:07:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:06:09.477287454 (25h37m45.516860564s) [upload=22556]: Starting upload Sep 7 15:06:09.477341047 (25h37m45.516914167s) [upload=22556]: Done starting upload Sep 7 15:06:09.477516443 (25h37m45.517089563s) [upload=22556]: Uploading segments... Sep 7 15:06:09.477550999 (25h37m45.517124109s) [upload=22556]: Got next segment. Inline: false Sep 7 15:06:09.477593901 (25h37m45.517167021s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:06:09.477598099 (25h37m45.517171219s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:06:09.763314436 (25h37m45.802887556s) [seg_pos={0 0}, upload=22556]: Begin upload segment... Sep 7 15:06:09.763361535 (25h37m45.802934655s) [seg_pos={0 0}, upload=22556]: Done begin upload segment. Sep 7 15:06:09.763384840 (25h37m45.80295796s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763420399 (25h37m45.802993519s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763812789 (25h37m45.803385919s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763990269 (25h37m45.803563389s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764372309 (25h37m45.80394543s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764475748 (25h37m45.804048868s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764744903 (25h37m45.804318023s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764786313 (25h37m45.804359433s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765207568 (25h37m45.804780688s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765475832 (25h37m45.805048952s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765740819 (25h37m45.805313939s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766083895 (25h37m45.805657015s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766410420 (25h37m45.80598357s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766463101 (25h37m45.806036211s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766743106 (25h37m45.806316226s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.767034234 (25h37m45.806607364s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:11.624806047 (25h37m47.664379167s) [upload=22556]: Next returned nil segment Sep 7 15:06:11.624840563 (25h37m47.664413693s) [upload=22556]: Waiting for error group managing segments... Sep 7 15:06:11.945497440 (25h37m47.98507056s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.948932752 (25h37m47.988505873s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.969614708 (25h37m48.009187818s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.990585857 (25h37m48.030158967s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.998861533 (25h37m48.038434653s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.003987873 (25h37m48.043560993s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.004947169 (25h37m48.044520299s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.010614082 (25h37m48.050187202s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.024736596 (25h37m48.064309716s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.036179973 (25h37m48.075753094s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.056595439 (25h37m48.096168559s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.083593591 (25h37m48.123166761s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094768284 (25h37m48.134341404s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094785115 (25h37m48.134358235s) [seg_pos={0 0}, upload=22556]: Segment reached optimal threshold of 13 pieces. Sep 7 15:06:12.094966282 (25h37m48.134539402s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095094788 (25h37m48.134667909s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection Sep 7 15:06:12.095123533 (25h37m48.134696643s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095165353 (25h37m48.134738473s) [seg_pos={0 0}, upload=22556]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:06:12.095253583 (25h37m48.134826703s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:06:12.095259113 (25h37m48.134832223s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:06:12.405199943 (25h37m48.444773134s) [upload=22556]: Done uploading segments... Sep 7 15:06:12.405225172 (25h37m48.444798302s) [upload=22556]: Upload finished. err: <nil> Duration: 13.382748638s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [.X.X...XX..X.....XXXX.X..XX..X.......XX.X.XXX......X....X..X] duration: 30.002640888s FAIL (2025-09-07 15:06:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:06:09.477287454 (25h37m45.516860564s) [upload=22556]: Starting upload Sep 7 15:06:09.477341047 (25h37m45.516914167s) [upload=22556]: Done starting upload Sep 7 15:06:09.477516443 (25h37m45.517089563s) [upload=22556]: Uploading segments... Sep 7 15:06:09.477550999 (25h37m45.517124109s) [upload=22556]: Got next segment. Inline: false Sep 7 15:06:09.477593901 (25h37m45.517167021s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:06:09.477598099 (25h37m45.517171219s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:06:09.763314436 (25h37m45.802887556s) [seg_pos={0 0}, upload=22556]: Begin upload segment... Sep 7 15:06:09.763361535 (25h37m45.802934655s) [seg_pos={0 0}, upload=22556]: Done begin upload segment. Sep 7 15:06:09.763384840 (25h37m45.80295796s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763420399 (25h37m45.802993519s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763812789 (25h37m45.803385919s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.763990269 (25h37m45.803563389s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764372309 (25h37m45.80394543s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764475748 (25h37m45.804048868s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764744903 (25h37m45.804318023s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.764786313 (25h37m45.804359433s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765207568 (25h37m45.804780688s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765475832 (25h37m45.805048952s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.765740819 (25h37m45.805313939s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766083895 (25h37m45.805657015s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766410420 (25h37m45.80598357s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766463101 (25h37m45.806036211s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.766743106 (25h37m45.806316226s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:09.767034234 (25h37m45.806607364s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Uploading piece... Sep 7 15:06:11.624806047 (25h37m47.664379167s) [upload=22556]: Next returned nil segment Sep 7 15:06:11.624840563 (25h37m47.664413693s) [upload=22556]: Waiting for error group managing segments... Sep 7 15:06:11.945497440 (25h37m47.98507056s) [noise=true , address=45.91.116.6:4974 , piece_id=O6H3U5QKEIGBGDF35EDGNPIPNJQKR3P33JKBMMJR4BEKZX7DE42A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.948932752 (25h37m47.988505873s) [noise=true , address=45.91.116.11:4119 , piece_id=MDKV5ICSWP3RVNWUVI3LQCXAQ5SETTXKFHBO7GBPQVFZZR5ACIRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.969614708 (25h37m48.009187818s) [noise=true , address=77.90.31.13:4627 , piece_id=YWT4XTOXR7BEWHFFHZ7YPFZOZKGSXITZICOXMSHROVBWX2222ILA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.990585857 (25h37m48.030158967s) [noise=true , address=45.91.116.14:4190 , piece_id=LRSMRE4U6ALFZOT37JGHGLU7QZOAKZ6ORFBGVFF334OE4DFRBXRQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:11.998861533 (25h37m48.038434653s) [noise=true , address=77.90.31.16:4734 , piece_id=3G5PDGDOIWFJVWW5AJ5TAPYKX2VVPC7RZQWHOZDL7GAAW277MERA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.003987873 (25h37m48.043560993s) [noise=true , address=45.91.116.4:4938 , piece_id=MMQHVIN6S2OZPBS5TIVN7ZQ43K6ZMYV4BMUQDELZHMTACJ2MNV5Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.004947169 (25h37m48.044520299s) [noise=true , address=45.91.116.5:4299 , piece_id=CPL52G5KROF5ZYXVJLA2M3SLA4WB6TZCVGN6ITDU7RDHL5BJ43DQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.010614082 (25h37m48.050187202s) [noise=true , address=77.90.31.15:4697 , piece_id=BXU3Q5F64GLNHAQPHDPDFSQB5QKBEODTVV2VZQRWKOD2VV6DANGA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.024736596 (25h37m48.064309716s) [noise=true , address=45.91.116.7:4009 , piece_id=MO7X7GRFGYUG5YT4X7SOL3MW5P463HEVFQFCGAGUMSLQYCMESWEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.036179973 (25h37m48.075753094s) [noise=true , address=77.90.31.19:4841 , piece_id=O77W5KB3GPGP5KI2TT3ZDQOFCJ5VGAERCA4YHJC3A25FB7IVIILQ, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.056595439 (25h37m48.096168559s) [noise=true , address=45.91.116.8:4047 , piece_id=2YJFW6ZU4E6DSHDQCPZI6EKVOBGZZG7NRJ2NLH2TRJVRSLPXK4GA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.083593591 (25h37m48.123166761s) [noise=true , address=45.91.116.12:4225 , piece_id=P2RYJCJ6Z6WNWVYCWO7DEZ7GHOT6IB3X2ZWATO6ANL2DR22DFKEA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094768284 (25h37m48.134341404s) [noise=true , address=45.91.116.6:4973 , piece_id=63WOLE5P2KKL3QNZ7LDIJHL5AEQ5UUXA7R6GOUCHKVR2BOWL2ZHA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: <nil> Sep 7 15:06:12.094785115 (25h37m48.134358235s) [seg_pos={0 0}, upload=22556]: Segment reached optimal threshold of 13 pieces. Sep 7 15:06:12.094966282 (25h37m48.134539402s) [noise=true , address=77.90.31.21:4878 , piece_id=XC65ZPV2UOKMBXWR26J27W4O6TA5J36HRBLVE77477KUZ3ZW6J3Q, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095094788 (25h37m48.134667909s) [noise=true , address=77.90.31.16:4735 , piece_id=KGABITY43IHH57TXLACZ2F6JTYGKQM7ZVZ625GXIFHHNTZCIEENA, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39626->77.90.31.16:4735: use of closed network connection Sep 7 15:06:12.095123533 (25h37m48.134696643s) [noise=true , address=45.91.116.12:4226 , piece_id=4N53NWDMSWWFFSA2HU2DXZAX553QVCU7GHPEBILO4PDFUHVTXU2A, seg_pos={0 0}, upload=22556]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 7 15:06:12.095165353 (25h37m48.134738473s) [seg_pos={0 0}, upload=22556]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:06:12.095253583 (25h37m48.134826703s) [upload=22556]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:06:12.095259113 (25h37m48.134832223s) [upload=22556]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:06:12.405199943 (25h37m48.444773134s) [upload=22556]: Done uploading segments... Sep 7 15:06:12.405225172 (25h37m48.444798302s) [upload=22556]: Upload finished. err: <nil> Duration: 30.00265722s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [..X.X...XX..X.....XXXX.X..XX..X.......XX.X.XXX......X....X..] duration: 24.119752975s FAIL (2025-09-07 15:05:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:03:41.299776291 (25h35m17.339349401s) [upload=22517]: Starting upload Sep 7 15:03:41.299835093 (25h35m17.339408213s) [upload=22517]: Done starting upload Sep 7 15:03:41.299920116 (25h35m17.339493236s) [upload=22517]: Uploading segments... Sep 7 15:03:41.300065333 (25h35m17.339638453s) [upload=22517]: Got next segment. Inline: false Sep 7 15:03:41.300094710 (25h35m17.33966783s) [upload=22517]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:03:41.300099229 (25h35m17.339672349s) [upload=22517]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:03:41.578344008 (25h35m17.617917128s) [seg_pos={0 0}, upload=22517]: Begin upload segment... Sep 7 15:03:41.578392782 (25h35m17.617965902s) [seg_pos={0 0}, upload=22517]: Done begin upload segment. Sep 7 15:03:41.578415405 (25h35m17.617988525s) [noise=true , address=45.91.116.7:4010 , piece_id=C3DTVQIUOEGIWHLDOVUB6OBZPEZLMRZ2DOVLWWGRS6FT3XL5GJBQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.578470901 (25h35m17.618044021s) [noise=true , address=77.90.31.18:4807 , piece_id=LME7CAKKLMQHP5DLJXE4W644WMBX76LDQSGSVJBZQ5E7WXP6DZOQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.578803778 (25h35m17.618376898s) [noise=true , address=45.91.116.10:4083 , piece_id=G7BRU65F334D4MBKUONCUXNBHTYL5CB24P4GHTGB3K46RMADLTCQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579029319 (25h35m17.618602439s) [noise=true , address=77.90.31.19:4842 , piece_id=LPR4B3U6UUKA67THFGBHPQK62BRSUS4I7EFXUYFXZ64XDWLFAO2Q, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579214474 (25h35m17.618787624s) [noise=true , address=45.91.116.6:4973 , piece_id=E4ELP4NX54OAC6JV5NAPCYYOPMFCRYI7MRZC7O46EJPGHHXBZY7Q, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579367497 (25h35m17.618940617s) [noise=true , address=45.91.116.13:4154 , piece_id=WPI5PRQAPUYQXGLN4CORGVLYLXRH4MEU6VGFV3OKBRTZCW76D6SQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579523315 (25h35m17.619096435s) [noise=true , address=45.91.116.5:4298 , piece_id=AZ3ZXSHCEQ7QQD55EGS4FJUHH3X57OKICYITPVOLERJGGIN6YKQQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579716074 (25h35m17.619289194s) [noise=true , address=45.91.116.8:4048 , piece_id=6SBX4ZDOB2YAKCT2BLGMRP2YU66MSXTX6NFFT4WCF7AOBXGOVZQQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579869708 (25h35m17.619442828s) [noise=true , address=45.91.116.9:4263 , piece_id=VRSFPCDA6PLQW7UNZFUVM5GSXPKDWTH723PDDNBANECRGPYVBGZQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580097363 (25h35m17.619670483s) [noise=true , address=45.91.116.11:4119 , piece_id=NBZCGDQYVD6LCZ6AYZDC7DXSNQLRZDLTRLBBA3HXRWXCV4FMGWIQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580307024 (25h35m17.619880145s) [noise=true , address=77.90.31.15:4697 , piece_id=PQH3HY6JNZMGLEC45MIPXQEUTMSI67UHVIDEZEHPYN4G3VCPJRFQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580514762 (25h35m17.620087882s) [noise=true , address=45.91.116.9:4261 , piece_id=2QX2GFAMMZA4PWX7UZD6TDQAYEVSZBTCY4K53DI627MWD2I2NUPQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580530502 (25h35m17.620103622s) [noise=true , address=45.91.116.8:4045 , piece_id=R3LLGSGLBIBVJGC2AKW2TIBCYJ7OTLKANYLXMNMNRDK5MDLL4HZA, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580748960 (25h35m17.62032208s) [noise=true , address=45.91.116.6:4974 , piece_id=JCTCLUJBNGVLMQD2FDXTWCZBVNPWKYPL4K2OVFYAVXJT2OPXGHZQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580870132 (25h35m17.620443252s) [noise=true , address=77.90.31.14:4662 , piece_id=RWE7NV2HSQUATJRQZYZJPHVCTEYKXYB5JVJUWS3UDQZLF6VFWPQQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.581082158 (25h35m17.620655278s) [noise=true , address=45.91.116.8:4047 , piece_id=BRXZXPRF25LKWQR7WFQ5KHNKLCLBQKP7JBYGYY3TT34JHXHX4NVA, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:43.486835805 (25h35m19.526408925s) [upload=22517]: Next returned nil segment Sep 7 15:03:43.486909376 (25h35m19.526482496s) [upload=22517]: Waiting for error group managing segments... Sep 7 15:03:43.799454852 (25h35m19.839027962s) [noise=true , address=45.91.116.8:4045 , piece_id=R3LLGSGLBIBVJGC2AKW2TIBCYJ7OTLKANYLXMNMNRDK5MDLL4HZA, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.811057344 (25h35m19.850630464s) [noise=true , address=45.91.116.7:4010 , piece_id=C3DTVQIUOEGIWHLDOVUB6OBZPEZLMRZ2DOVLWWGRS6FT3XL5GJBQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.829361429 (25h35m19.868934549s) [noise=true , address=77.90.31.14:4662 , piece_id=RWE7NV2HSQUATJRQZYZJPHVCTEYKXYB5JVJUWS3UDQZLF6VFWPQQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.830090995 (25h35m19.869664115s) [noise=true , address=45.91.116.9:4261 , piece_id=2QX2GFAMMZA4PWX7UZD6TDQAYEVSZBTCY4K53DI627MWD2I2NUPQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.834401192 (25h35m19.873974302s) [noise=true , address=45.91.116.13:4154 , piece_id=WPI5PRQAPUYQXGLN4CORGVLYLXRH4MEU6VGFV3OKBRTZCW76D6SQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.860747567 (25h35m19.900320687s) [noise=true , address=77.90.31.18:4807 , piece_id=LME7CAKKLMQHP5DLJXE4W644WMBX76LDQSGSVJBZQ5E7WXP6DZOQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.867782479 (25h35m19.907355589s) [noise=true , address=45.91.116.8:4047 , piece_id=BRXZXPRF25LKWQR7WFQ5KHNKLCLBQKP7JBYGYY3TT34JHXHX4NVA, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.876692899 (25h35m19.916266019s) [noise=true , address=45.91.116.6:4974 , piece_id=JCTCLUJBNGVLMQD2FDXTWCZBVNPWKYPL4K2OVFYAVXJT2OPXGHZQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.891822450 (25h35m19.93139557s) [noise=true , address=45.91.116.9:4263 , piece_id=VRSFPCDA6PLQW7UNZFUVM5GSXPKDWTH723PDDNBANECRGPYVBGZQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.894669458 (25h35m19.934242578s) [noise=true , address=77.90.31.15:4697 , piece_id=PQH3HY6JNZMGLEC45MIPXQEUTMSI67UHVIDEZEHPYN4G3VCPJRFQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.917086142 (25h35m19.956659262s) [noise=true , address=45.91.116.5:4298 , piece_id=AZ3ZXSHCEQ7QQD55EGS4FJUHH3X57OKICYITPVOLERJGGIN6YKQQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.930224734 (25h35m19.969797854s) [noise=true , address=45.91.116.8:4048 , piece_id=6SBX4ZDOB2YAKCT2BLGMRP2YU66MSXTX6NFFT4WCF7AOBXGOVZQQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.976171274 (25h35m20.015744394s) [noise=true , address=77.90.31.19:4842 , piece_id=LPR4B3U6UUKA67THFGBHPQK62BRSUS4I7EFXUYFXZ64XDWLFAO2Q, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.976190991 (25h35m20.015764111s) [seg_pos={0 0}, upload=22517]: Segment reached optimal threshold of 13 pieces. Sep 7 15:03:43.976431201 (25h35m20.016004321s) [noise=true , address=45.91.116.11:4119 , piece_id=NBZCGDQYVD6LCZ6AYZDC7DXSNQLRZDLTRLBBA3HXRWXCV4FMGWIQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: context canceled Sep 7 15:03:43.976460517 (25h35m20.016033637s) [noise=true , address=45.91.116.6:4973 , piece_id=E4ELP4NX54OAC6JV5NAPCYYOPMFCRYI7MRZC7O46EJPGHHXBZY7Q, seg_pos={0 0}, upload=22517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Sep 7 15:03:43.976518257 (25h35m20.016091377s) [noise=true , address=45.91.116.10:4083 , piece_id=G7BRU65F334D4MBKUONCUXNBHTYL5CB24P4GHTGB3K46RMADLTCQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:44992->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:44992->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44992->45.91.116.10:4083: use of closed network connection Sep 7 15:03:43.976542273 (25h35m20.016115393s) [seg_pos={0 0}, upload=22517]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:03:43.976607749 (25h35m20.016180869s) [upload=22517]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:03:43.976612128 (25h35m20.016185248s) [upload=22517]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:03:44.705173192 (25h35m20.744746312s) [upload=22517]: Done uploading segments... Sep 7 15:03:44.705192449 (25h35m20.74476557s) [upload=22517]: Upload finished. err: <nil> Duration: 24.119780807s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [X..X.X...XX..X.....XXXX.X..XX..X.......XX.X.XXX......X....X.] duration: 20.771730944s FAIL (2025-09-07 15:04:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:03:41.299776291 (25h35m17.339349401s) [upload=22517]: Starting upload Sep 7 15:03:41.299835093 (25h35m17.339408213s) [upload=22517]: Done starting upload Sep 7 15:03:41.299920116 (25h35m17.339493236s) [upload=22517]: Uploading segments... Sep 7 15:03:41.300065333 (25h35m17.339638453s) [upload=22517]: Got next segment. Inline: false Sep 7 15:03:41.300094710 (25h35m17.33966783s) [upload=22517]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:03:41.300099229 (25h35m17.339672349s) [upload=22517]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:03:41.578344008 (25h35m17.617917128s) [seg_pos={0 0}, upload=22517]: Begin upload segment... Sep 7 15:03:41.578392782 (25h35m17.617965902s) [seg_pos={0 0}, upload=22517]: Done begin upload segment. Sep 7 15:03:41.578415405 (25h35m17.617988525s) [noise=true , address=45.91.116.7:4010 , piece_id=C3DTVQIUOEGIWHLDOVUB6OBZPEZLMRZ2DOVLWWGRS6FT3XL5GJBQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.578470901 (25h35m17.618044021s) [noise=true , address=77.90.31.18:4807 , piece_id=LME7CAKKLMQHP5DLJXE4W644WMBX76LDQSGSVJBZQ5E7WXP6DZOQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.578803778 (25h35m17.618376898s) [noise=true , address=45.91.116.10:4083 , piece_id=G7BRU65F334D4MBKUONCUXNBHTYL5CB24P4GHTGB3K46RMADLTCQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579029319 (25h35m17.618602439s) [noise=true , address=77.90.31.19:4842 , piece_id=LPR4B3U6UUKA67THFGBHPQK62BRSUS4I7EFXUYFXZ64XDWLFAO2Q, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579214474 (25h35m17.618787624s) [noise=true , address=45.91.116.6:4973 , piece_id=E4ELP4NX54OAC6JV5NAPCYYOPMFCRYI7MRZC7O46EJPGHHXBZY7Q, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579367497 (25h35m17.618940617s) [noise=true , address=45.91.116.13:4154 , piece_id=WPI5PRQAPUYQXGLN4CORGVLYLXRH4MEU6VGFV3OKBRTZCW76D6SQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579523315 (25h35m17.619096435s) [noise=true , address=45.91.116.5:4298 , piece_id=AZ3ZXSHCEQ7QQD55EGS4FJUHH3X57OKICYITPVOLERJGGIN6YKQQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579716074 (25h35m17.619289194s) [noise=true , address=45.91.116.8:4048 , piece_id=6SBX4ZDOB2YAKCT2BLGMRP2YU66MSXTX6NFFT4WCF7AOBXGOVZQQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579869708 (25h35m17.619442828s) [noise=true , address=45.91.116.9:4263 , piece_id=VRSFPCDA6PLQW7UNZFUVM5GSXPKDWTH723PDDNBANECRGPYVBGZQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580097363 (25h35m17.619670483s) [noise=true , address=45.91.116.11:4119 , piece_id=NBZCGDQYVD6LCZ6AYZDC7DXSNQLRZDLTRLBBA3HXRWXCV4FMGWIQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580307024 (25h35m17.619880145s) [noise=true , address=77.90.31.15:4697 , piece_id=PQH3HY6JNZMGLEC45MIPXQEUTMSI67UHVIDEZEHPYN4G3VCPJRFQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580514762 (25h35m17.620087882s) [noise=true , address=45.91.116.9:4261 , piece_id=2QX2GFAMMZA4PWX7UZD6TDQAYEVSZBTCY4K53DI627MWD2I2NUPQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580530502 (25h35m17.620103622s) [noise=true , address=45.91.116.8:4045 , piece_id=R3LLGSGLBIBVJGC2AKW2TIBCYJ7OTLKANYLXMNMNRDK5MDLL4HZA, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580748960 (25h35m17.62032208s) [noise=true , address=45.91.116.6:4974 , piece_id=JCTCLUJBNGVLMQD2FDXTWCZBVNPWKYPL4K2OVFYAVXJT2OPXGHZQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580870132 (25h35m17.620443252s) [noise=true , address=77.90.31.14:4662 , piece_id=RWE7NV2HSQUATJRQZYZJPHVCTEYKXYB5JVJUWS3UDQZLF6VFWPQQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.581082158 (25h35m17.620655278s) [noise=true , address=45.91.116.8:4047 , piece_id=BRXZXPRF25LKWQR7WFQ5KHNKLCLBQKP7JBYGYY3TT34JHXHX4NVA, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:43.486835805 (25h35m19.526408925s) [upload=22517]: Next returned nil segment Sep 7 15:03:43.486909376 (25h35m19.526482496s) [upload=22517]: Waiting for error group managing segments... Sep 7 15:03:43.799454852 (25h35m19.839027962s) [noise=true , address=45.91.116.8:4045 , piece_id=R3LLGSGLBIBVJGC2AKW2TIBCYJ7OTLKANYLXMNMNRDK5MDLL4HZA, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.811057344 (25h35m19.850630464s) [noise=true , address=45.91.116.7:4010 , piece_id=C3DTVQIUOEGIWHLDOVUB6OBZPEZLMRZ2DOVLWWGRS6FT3XL5GJBQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.829361429 (25h35m19.868934549s) [noise=true , address=77.90.31.14:4662 , piece_id=RWE7NV2HSQUATJRQZYZJPHVCTEYKXYB5JVJUWS3UDQZLF6VFWPQQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.830090995 (25h35m19.869664115s) [noise=true , address=45.91.116.9:4261 , piece_id=2QX2GFAMMZA4PWX7UZD6TDQAYEVSZBTCY4K53DI627MWD2I2NUPQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.834401192 (25h35m19.873974302s) [noise=true , address=45.91.116.13:4154 , piece_id=WPI5PRQAPUYQXGLN4CORGVLYLXRH4MEU6VGFV3OKBRTZCW76D6SQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.860747567 (25h35m19.900320687s) [noise=true , address=77.90.31.18:4807 , piece_id=LME7CAKKLMQHP5DLJXE4W644WMBX76LDQSGSVJBZQ5E7WXP6DZOQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.867782479 (25h35m19.907355589s) [noise=true , address=45.91.116.8:4047 , piece_id=BRXZXPRF25LKWQR7WFQ5KHNKLCLBQKP7JBYGYY3TT34JHXHX4NVA, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.876692899 (25h35m19.916266019s) [noise=true , address=45.91.116.6:4974 , piece_id=JCTCLUJBNGVLMQD2FDXTWCZBVNPWKYPL4K2OVFYAVXJT2OPXGHZQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.891822450 (25h35m19.93139557s) [noise=true , address=45.91.116.9:4263 , piece_id=VRSFPCDA6PLQW7UNZFUVM5GSXPKDWTH723PDDNBANECRGPYVBGZQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.894669458 (25h35m19.934242578s) [noise=true , address=77.90.31.15:4697 , piece_id=PQH3HY6JNZMGLEC45MIPXQEUTMSI67UHVIDEZEHPYN4G3VCPJRFQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.917086142 (25h35m19.956659262s) [noise=true , address=45.91.116.5:4298 , piece_id=AZ3ZXSHCEQ7QQD55EGS4FJUHH3X57OKICYITPVOLERJGGIN6YKQQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.930224734 (25h35m19.969797854s) [noise=true , address=45.91.116.8:4048 , piece_id=6SBX4ZDOB2YAKCT2BLGMRP2YU66MSXTX6NFFT4WCF7AOBXGOVZQQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.976171274 (25h35m20.015744394s) [noise=true , address=77.90.31.19:4842 , piece_id=LPR4B3U6UUKA67THFGBHPQK62BRSUS4I7EFXUYFXZ64XDWLFAO2Q, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.976190991 (25h35m20.015764111s) [seg_pos={0 0}, upload=22517]: Segment reached optimal threshold of 13 pieces. Sep 7 15:03:43.976431201 (25h35m20.016004321s) [noise=true , address=45.91.116.11:4119 , piece_id=NBZCGDQYVD6LCZ6AYZDC7DXSNQLRZDLTRLBBA3HXRWXCV4FMGWIQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: context canceled Sep 7 15:03:43.976460517 (25h35m20.016033637s) [noise=true , address=45.91.116.6:4973 , piece_id=E4ELP4NX54OAC6JV5NAPCYYOPMFCRYI7MRZC7O46EJPGHHXBZY7Q, seg_pos={0 0}, upload=22517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Sep 7 15:03:43.976518257 (25h35m20.016091377s) [noise=true , address=45.91.116.10:4083 , piece_id=G7BRU65F334D4MBKUONCUXNBHTYL5CB24P4GHTGB3K46RMADLTCQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:44992->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:44992->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44992->45.91.116.10:4083: use of closed network connection Sep 7 15:03:43.976542273 (25h35m20.016115393s) [seg_pos={0 0}, upload=22517]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:03:43.976607749 (25h35m20.016180869s) [upload=22517]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:03:43.976612128 (25h35m20.016185248s) [upload=22517]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:03:44.705173192 (25h35m20.744746312s) [upload=22517]: Done uploading segments... Sep 7 15:03:44.705192449 (25h35m20.74476557s) [upload=22517]: Upload finished. err: <nil> Duration: 20.771748697s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [XX..X.X...XX..X.....XXXX.X..XX..X.......XX.X.XXX......X....X] duration: 30.002253078s FAIL (2025-09-07 15:04:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 15:03:41.299776291 (25h35m17.339349401s) [upload=22517]: Starting upload Sep 7 15:03:41.299835093 (25h35m17.339408213s) [upload=22517]: Done starting upload Sep 7 15:03:41.299920116 (25h35m17.339493236s) [upload=22517]: Uploading segments... Sep 7 15:03:41.300065333 (25h35m17.339638453s) [upload=22517]: Got next segment. Inline: false Sep 7 15:03:41.300094710 (25h35m17.33966783s) [upload=22517]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 15:03:41.300099229 (25h35m17.339672349s) [upload=22517]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 15:03:41.578344008 (25h35m17.617917128s) [seg_pos={0 0}, upload=22517]: Begin upload segment... Sep 7 15:03:41.578392782 (25h35m17.617965902s) [seg_pos={0 0}, upload=22517]: Done begin upload segment. Sep 7 15:03:41.578415405 (25h35m17.617988525s) [noise=true , address=45.91.116.7:4010 , piece_id=C3DTVQIUOEGIWHLDOVUB6OBZPEZLMRZ2DOVLWWGRS6FT3XL5GJBQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.578470901 (25h35m17.618044021s) [noise=true , address=77.90.31.18:4807 , piece_id=LME7CAKKLMQHP5DLJXE4W644WMBX76LDQSGSVJBZQ5E7WXP6DZOQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.578803778 (25h35m17.618376898s) [noise=true , address=45.91.116.10:4083 , piece_id=G7BRU65F334D4MBKUONCUXNBHTYL5CB24P4GHTGB3K46RMADLTCQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579029319 (25h35m17.618602439s) [noise=true , address=77.90.31.19:4842 , piece_id=LPR4B3U6UUKA67THFGBHPQK62BRSUS4I7EFXUYFXZ64XDWLFAO2Q, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579214474 (25h35m17.618787624s) [noise=true , address=45.91.116.6:4973 , piece_id=E4ELP4NX54OAC6JV5NAPCYYOPMFCRYI7MRZC7O46EJPGHHXBZY7Q, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579367497 (25h35m17.618940617s) [noise=true , address=45.91.116.13:4154 , piece_id=WPI5PRQAPUYQXGLN4CORGVLYLXRH4MEU6VGFV3OKBRTZCW76D6SQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579523315 (25h35m17.619096435s) [noise=true , address=45.91.116.5:4298 , piece_id=AZ3ZXSHCEQ7QQD55EGS4FJUHH3X57OKICYITPVOLERJGGIN6YKQQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579716074 (25h35m17.619289194s) [noise=true , address=45.91.116.8:4048 , piece_id=6SBX4ZDOB2YAKCT2BLGMRP2YU66MSXTX6NFFT4WCF7AOBXGOVZQQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.579869708 (25h35m17.619442828s) [noise=true , address=45.91.116.9:4263 , piece_id=VRSFPCDA6PLQW7UNZFUVM5GSXPKDWTH723PDDNBANECRGPYVBGZQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580097363 (25h35m17.619670483s) [noise=true , address=45.91.116.11:4119 , piece_id=NBZCGDQYVD6LCZ6AYZDC7DXSNQLRZDLTRLBBA3HXRWXCV4FMGWIQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580307024 (25h35m17.619880145s) [noise=true , address=77.90.31.15:4697 , piece_id=PQH3HY6JNZMGLEC45MIPXQEUTMSI67UHVIDEZEHPYN4G3VCPJRFQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580514762 (25h35m17.620087882s) [noise=true , address=45.91.116.9:4261 , piece_id=2QX2GFAMMZA4PWX7UZD6TDQAYEVSZBTCY4K53DI627MWD2I2NUPQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580530502 (25h35m17.620103622s) [noise=true , address=45.91.116.8:4045 , piece_id=R3LLGSGLBIBVJGC2AKW2TIBCYJ7OTLKANYLXMNMNRDK5MDLL4HZA, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580748960 (25h35m17.62032208s) [noise=true , address=45.91.116.6:4974 , piece_id=JCTCLUJBNGVLMQD2FDXTWCZBVNPWKYPL4K2OVFYAVXJT2OPXGHZQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.580870132 (25h35m17.620443252s) [noise=true , address=77.90.31.14:4662 , piece_id=RWE7NV2HSQUATJRQZYZJPHVCTEYKXYB5JVJUWS3UDQZLF6VFWPQQ, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:41.581082158 (25h35m17.620655278s) [noise=true , address=45.91.116.8:4047 , piece_id=BRXZXPRF25LKWQR7WFQ5KHNKLCLBQKP7JBYGYY3TT34JHXHX4NVA, seg_pos={0 0}, upload=22517]: Uploading piece... Sep 7 15:03:43.486835805 (25h35m19.526408925s) [upload=22517]: Next returned nil segment Sep 7 15:03:43.486909376 (25h35m19.526482496s) [upload=22517]: Waiting for error group managing segments... Sep 7 15:03:43.799454852 (25h35m19.839027962s) [noise=true , address=45.91.116.8:4045 , piece_id=R3LLGSGLBIBVJGC2AKW2TIBCYJ7OTLKANYLXMNMNRDK5MDLL4HZA, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.811057344 (25h35m19.850630464s) [noise=true , address=45.91.116.7:4010 , piece_id=C3DTVQIUOEGIWHLDOVUB6OBZPEZLMRZ2DOVLWWGRS6FT3XL5GJBQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.829361429 (25h35m19.868934549s) [noise=true , address=77.90.31.14:4662 , piece_id=RWE7NV2HSQUATJRQZYZJPHVCTEYKXYB5JVJUWS3UDQZLF6VFWPQQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.830090995 (25h35m19.869664115s) [noise=true , address=45.91.116.9:4261 , piece_id=2QX2GFAMMZA4PWX7UZD6TDQAYEVSZBTCY4K53DI627MWD2I2NUPQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.834401192 (25h35m19.873974302s) [noise=true , address=45.91.116.13:4154 , piece_id=WPI5PRQAPUYQXGLN4CORGVLYLXRH4MEU6VGFV3OKBRTZCW76D6SQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.860747567 (25h35m19.900320687s) [noise=true , address=77.90.31.18:4807 , piece_id=LME7CAKKLMQHP5DLJXE4W644WMBX76LDQSGSVJBZQ5E7WXP6DZOQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.867782479 (25h35m19.907355589s) [noise=true , address=45.91.116.8:4047 , piece_id=BRXZXPRF25LKWQR7WFQ5KHNKLCLBQKP7JBYGYY3TT34JHXHX4NVA, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.876692899 (25h35m19.916266019s) [noise=true , address=45.91.116.6:4974 , piece_id=JCTCLUJBNGVLMQD2FDXTWCZBVNPWKYPL4K2OVFYAVXJT2OPXGHZQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.891822450 (25h35m19.93139557s) [noise=true , address=45.91.116.9:4263 , piece_id=VRSFPCDA6PLQW7UNZFUVM5GSXPKDWTH723PDDNBANECRGPYVBGZQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.894669458 (25h35m19.934242578s) [noise=true , address=77.90.31.15:4697 , piece_id=PQH3HY6JNZMGLEC45MIPXQEUTMSI67UHVIDEZEHPYN4G3VCPJRFQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.917086142 (25h35m19.956659262s) [noise=true , address=45.91.116.5:4298 , piece_id=AZ3ZXSHCEQ7QQD55EGS4FJUHH3X57OKICYITPVOLERJGGIN6YKQQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.930224734 (25h35m19.969797854s) [noise=true , address=45.91.116.8:4048 , piece_id=6SBX4ZDOB2YAKCT2BLGMRP2YU66MSXTX6NFFT4WCF7AOBXGOVZQQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.976171274 (25h35m20.015744394s) [noise=true , address=77.90.31.19:4842 , piece_id=LPR4B3U6UUKA67THFGBHPQK62BRSUS4I7EFXUYFXZ64XDWLFAO2Q, seg_pos={0 0}, upload=22517]: Done uploading piece. err: <nil> Sep 7 15:03:43.976190991 (25h35m20.015764111s) [seg_pos={0 0}, upload=22517]: Segment reached optimal threshold of 13 pieces. Sep 7 15:03:43.976431201 (25h35m20.016004321s) [noise=true , address=45.91.116.11:4119 , piece_id=NBZCGDQYVD6LCZ6AYZDC7DXSNQLRZDLTRLBBA3HXRWXCV4FMGWIQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: context canceled Sep 7 15:03:43.976460517 (25h35m20.016033637s) [noise=true , address=45.91.116.6:4973 , piece_id=E4ELP4NX54OAC6JV5NAPCYYOPMFCRYI7MRZC7O46EJPGHHXBZY7Q, seg_pos={0 0}, upload=22517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Sep 7 15:03:43.976518257 (25h35m20.016091377s) [noise=true , address=45.91.116.10:4083 , piece_id=G7BRU65F334D4MBKUONCUXNBHTYL5CB24P4GHTGB3K46RMADLTCQ, seg_pos={0 0}, upload=22517]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:44992->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:44992->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44992->45.91.116.10:4083: use of closed network connection Sep 7 15:03:43.976542273 (25h35m20.016115393s) [seg_pos={0 0}, upload=22517]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 15:03:43.976607749 (25h35m20.016180869s) [upload=22517]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 15:03:43.976612128 (25h35m20.016185248s) [upload=22517]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 15:03:44.705173192 (25h35m20.744746312s) [upload=22517]: Done uploading segments... Sep 7 15:03:44.705192449 (25h35m20.74476557s) [upload=22517]: Upload finished. err: <nil> Duration: 30.002277545s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)