Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.856327641s FAIL (2025-09-08 01:18:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 5.856364089s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.830170342s FAIL (2025-09-08 01:17:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 5.830196334s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.96854843s FAIL (2025-09-08 01:17:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.968583226s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.237311454s FAIL (2025-09-08 01:17:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.237346894s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.927099957s FAIL (2025-09-08 01:17:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.92713276s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.877978978s FAIL (2025-09-08 01:16:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 5.878008304s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.016817816s FAIL (2025-09-08 01:16:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.016840961s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.977491833s FAIL (2025-09-08 01:16:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.977512092s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.484279084s FAIL (2025-09-08 01:15:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.48430208s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.165355551s FAIL (2025-09-08 01:15:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.165384357s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.133428345s FAIL (2025-09-08 01:14:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.13345706s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.307442854s FAIL (2025-09-08 01:14:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.307470579s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.037441197s FAIL (2025-09-08 01:13:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.037470542s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.112745906s FAIL (2025-09-08 01:13:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.112773258s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.24733299s FAIL (2025-09-08 01:13:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.247361855s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 15.778392242s FAIL (2025-09-08 01:13:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 15.778422631s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 17.790027975s FAIL (2025-09-08 01:12:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 17.790059766s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 14.782058632s FAIL (2025-09-08 01:12:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 14.782086487s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.037239349s FAIL (2025-09-08 01:12:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.037271512s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.044651955s FAIL (2025-09-08 01:11:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.044678807s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.001514819s FAIL (2025-09-08 01:10:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.001627454s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.932182772s FAIL (2025-09-08 01:10:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.932205244s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.364269723s FAIL (2025-09-08 01:10:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.364305501s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.864074647s FAIL (2025-09-08 01:10:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 5.864094204s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.093132532s FAIL (2025-09-08 01:09:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.093157742s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.184019211s FAIL (2025-09-08 01:09:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.184044588s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.086501061s FAIL (2025-09-08 01:08:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.086534125s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.246811354s FAIL (2025-09-08 01:08:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.24684038s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.913975545s FAIL (2025-09-08 01:08:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 5.914005722s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.84159856s FAIL (2025-09-08 01:07:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.841627505s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.024763928s FAIL (2025-09-08 01:07:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 6.024795398s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.861140824s FAIL (2025-09-08 01:06:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 5.861167253s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.888458029s FAIL (2025-09-08 01:06:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 5.888477837s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.10925379s FAIL (2025-09-08 01:06:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.109286663s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 19.18229908s FAIL (2025-09-08 01:06:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 19.182326942s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 13.658152948s FAIL (2025-09-08 01:06:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 13.65817507s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.002163855s FAIL (2025-09-08 01:06:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 01:05:04.294020248 (35h36m40.333593368s) [upload=31295]: Starting upload Sep 8 01:05:04.294084921 (35h36m40.333658051s) [upload=31295]: Done starting upload Sep 8 01:05:04.294247722 (35h36m40.333820843s) [upload=31295]: Uploading segments... Sep 8 01:05:04.294382120 (35h36m40.33395524s) [upload=31295]: Got next segment. Inline: false Sep 8 01:05:04.294425233 (35h36m40.333998353s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:04.294430493 (35h36m40.334003613s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:04.866639830 (35h36m40.90621295s) [seg_pos={0 0}, upload=31295]: Begin upload segment... Sep 8 01:05:04.866689444 (35h36m40.906262564s) [seg_pos={0 0}, upload=31295]: Done begin upload segment. Sep 8 01:05:04.866714362 (35h36m40.906287482s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.866795087 (35h36m40.906368217s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867098647 (35h36m40.906671767s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867452745 (35h36m40.907025865s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867493333 (35h36m40.907066453s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867744964 (35h36m40.907318084s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.867895722 (35h36m40.907468842s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868025070 (35h36m40.9075982s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868298654 (35h36m40.907871784s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868314333 (35h36m40.907887453s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868345644 (35h36m40.907918764s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868574091 (35h36m40.908147211s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868676527 (35h36m40.908249647s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.868849467 (35h36m40.908422588s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869075991 (35h36m40.908649111s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869105667 (35h36m40.908678787s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869340898 (35h36m40.908914018s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:04.869452391 (35h36m40.909025511s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Uploading piece... Sep 8 01:05:06.395648000 (35h36m42.43522112s) [upload=31295]: Next returned nil segment Sep 8 01:05:06.395664983 (35h36m42.435238103s) [upload=31295]: Waiting for error group managing segments... Sep 8 01:05:06.790549857 (35h36m42.830122977s) [noise=true , address=145.79.227.4:4214 , piece_id=U5TRBOHRL2BAAM4A5S32YWJVF6XG2IHTKZKGAWTCB43B2SUMCCEQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.807695422 (35h36m42.847268542s) [noise=true , address=145.79.227.7:4303 , piece_id=AGSLJHU44JFJ26DRI4MSVKFHKCNCG5JPC656KQWUAACMFLDULWTA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.853444697 (35h36m42.893017817s) [noise=true , address=145.79.227.49:4339 , piece_id=CKUK5DZ4GMRYKR4SVVG4TMHUDRL26PB36WP6BE36MR52WDQ7FZCQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.866919158 (35h36m42.906492278s) [noise=true , address=145.79.227.6:4261 , piece_id=PCRBPC2FJY2SUU2PQI6D6C2B4UTO666YOIL7V4AGET4GS72OG6PA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.882171311 (35h36m42.921744431s) [noise=true , address=77.90.31.17:4785 , piece_id=TYR5SDV3BZMYIWYZMK2A6NGECSDFM2HUA3P3ZHXV6PI77M6ZD4HA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.900525018 (35h36m42.940098138s) [noise=true , address=145.79.227.47:4265 , piece_id=SD4Y2NSBSZTYA7WXK6NA7WB3KQFQDNR5KG5NIU4GT3SHHQEPMEXA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.904741876 (35h36m42.944314997s) [noise=true , address=145.79.227.38:4994 , piece_id=UNGTAICMB57HO5JEN76UBAU4OWUYDCTPKCNL53CTCXFYFDNP6CSQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.907456490 (35h36m42.94702961s) [noise=true , address=145.79.227.42:4121 , piece_id=WOQCCOI3KEXO6BX6M2NAA3GIOH7AKY7PIV6J5VKOLM4TKYG2M24Q, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.911368315 (35h36m42.950941445s) [noise=true , address=145.79.227.43:4183 , piece_id=TGUWLCJPLM4KNZWUIV6BN7GHSBI7RUITCFNWQJNDH4QWS5575VRQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.925378681 (35h36m42.964951841s) [noise=true , address=145.79.227.18:4031 , piece_id=5CBZ6G5GEZDAONDHCAQ2C43PSJQHLDMDQJXRDAOTFXQHUUAOIOPA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.928129575 (35h36m42.967702695s) [noise=true , address=145.79.227.37:4594 , piece_id=YBZ65ZVXT5TRTMUKUOPBSI5ED7XUDPFJMYZ2SN3GNA73NRFPE5ZQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.937432024 (35h36m42.977005144s) [noise=true , address=145.79.227.14:4845 , piece_id=2KKOOINYEZTZGBTTNV3NAL4LWJ7AUPBWOJHLFHXM4W5SSRIQ4QZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.938572887 (35h36m42.978146007s) [noise=true , address=145.79.227.33:4895 , piece_id=U5GXBZAURUDHR677GV6VHUVALA6TRHV4F6GSUO23AUE22WPALTJQ, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966722898 (35h36m43.006296018s) [noise=true , address=145.79.227.17:4975 , piece_id=J5TSDS2NM6Y35KKXIWOE4NRWDVKU3AQX2OVIIIFMP3AJXWDJJLZA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: <nil> Sep 8 01:05:06.966753977 (35h36m43.006327097s) [seg_pos={0 0}, upload=31295]: Segment reached optimal threshold of 14 pieces. Sep 8 01:05:06.966909785 (35h36m43.006482905s) [noise=true , address=145.79.227.15:4887 , piece_id=YO2KJYJZT7XJY7IUUS6RSTVBJ4HKD2LF63X7LVG5CNXOJDFKT6IA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19QrU8gBXbSzy9LzhkHTYgdtTbZG51wV9CSWRcvvQuM6Jex9d7): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.966975080 (35h36m43.0065482s) [noise=true , address=145.79.227.42:4138 , piece_id=IZROO6FDSA7EBWDNPGWCE2KQ2XEQYAWT5KF55PDFBM3UQWXKJG6A, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967018092 (35h36m43.006591212s) [noise=true , address=145.79.227.41:4120 , piece_id=KIWCAOPQJJTDBS5NWAJ3SIQIX6FIDN6AIURL6XKCCPX7TDKEXDLA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aKMhWsxNocDaqpuPLy7177jEA9K6MrsuKi1awcDcod3mM1vM3): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967063319 (35h36m43.006636439s) [noise=true , address=145.79.227.21:4477 , piece_id=J2JAPZ3IBJJJXTCHZDG57265GTAPDANTSPLMSV75RATXY754LHQA, seg_pos={0 0}, upload=31295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY): protocol: expected piece hash; context canceled; EOF Sep 8 01:05:06.967076985 (35h36m43.006650095s) [seg_pos={0 0}, upload=31295]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 8 01:05:06.967162378 (35h36m43.006735499s) [upload=31295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 01:05:06.967168179 (35h36m43.006741299s) [upload=31295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 01:05:07.310679357 (35h36m43.350252477s) [upload=31295]: Done uploading segments... Sep 8 01:05:07.310700318 (35h36m43.350273428s) [upload=31295]: Upload finished. err: <nil> Duration: 1m0.002189926s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.267689093s FAIL (2025-09-08 01:05:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 8 01:05:12.757110512 (35h36m48.796683632s) [upload=31297]: Starting upload Sep 8 01:05:12.757174896 (35h36m48.796748016s) [upload=31297]: Done starting upload Sep 8 01:05:12.757256542 (35h36m48.796829662s) [upload=31297]: Uploading segments... Sep 8 01:05:12.757358648 (35h36m48.796931768s) [upload=31297]: Got next segment. Inline: false Sep 8 01:05:12.757392763 (35h36m48.796965883s) [upload=31297]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 01:05:12.757397031 (35h36m48.796970151s) [upload=31297]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 01:05:13.092807249 (35h36m49.132380369s) [seg_pos={0 0}, upload=31297]: Begin upload segment... Sep 8 01:05:13.092850261 (35h36m49.132423381s) [seg_pos={0 0}, upload=31297]: Done begin upload segment. Sep 8 01:05:13.092877734 (35h36m49.132450854s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.092884446 (35h36m49.132457567s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093230309 (35h36m49.132803429s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093271006 (35h36m49.132844126s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093493782 (35h36m49.133066902s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.093777766 (35h36m49.133350886s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094080435 (35h36m49.133653555s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094352857 (35h36m49.133925977s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094365460 (35h36m49.13393858s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094578719 (35h36m49.134151839s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094790153 (35h36m49.134363273s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.094951651 (35h36m49.134524772s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095364361 (35h36m49.134937481s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.095717928 (35h36m49.135291058s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096012852 (35h36m49.135585972s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:13.096260797 (35h36m49.135833927s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Uploading piece... Sep 8 01:05:14.873078281 (35h36m50.912651401s) [upload=31297]: Next returned nil segment Sep 8 01:05:14.873094051 (35h36m50.912667162s) [upload=31297]: Waiting for error group managing segments... Sep 8 01:05:15.202698280 (35h36m51.2422714s) [noise=true , address=45.91.116.5:4297 , piece_id=NHLHOA5HF5JR5JB2V2LCS44QN25SEDKT3BYWU446IW524U6H65HQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.209094027 (35h36m51.248667147s) [noise=true , address=77.90.31.15:4699 , piece_id=MC4RD2DNSBTA6GS4ZXZS6VO2BWHV7UBUWPNGGZVWWQJAUHMADBPQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.212692733 (35h36m51.252265853s) [noise=true , address=45.91.116.14:4189 , piece_id=IJYRXOOZSTU3NU2UNFNXSJI22FY2T7ZDJXJWUGDITAJTGAXGP3JQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.248887306 (35h36m51.288460426s) [noise=true , address=45.91.116.12:4225 , piece_id=UPPD3VNT43L2IQHFPCXGUGFGV36IWC2S3KI4JCUSCYOWDRVCW6VQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.254327434 (35h36m51.293900554s) [noise=true , address=45.91.116.6:4973 , piece_id=XMG3Q3IOTLSSKYGOQO6HOVBFF2XJ5T7NTKCUWVQ43AXJULRMMXHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.255605851 (35h36m51.295178971s) [noise=true , address=77.90.31.18:4806 , piece_id=5J5XUUNTCKDP7NAEOY7S5Z7AKGNSIRPPVDJUHS3SNWXIUTYLO6WA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265299779 (35h36m51.304872899s) [noise=true , address=77.90.31.17:4772 , piece_id=M3YW2HBA7FDYO7L5GZV4REBIFWULTAW5NHAG46XZPGMRDQCXWHHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.265911729 (35h36m51.305484849s) [noise=true , address=77.90.31.14:4663 , piece_id=YJEC7FCENEJKH62OHDSKPFKMJS3WMJCG5IYEHLHOZASJUKM3G7SQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.276901737 (35h36m51.316474857s) [noise=true , address=45.91.116.9:4263 , piece_id=QTVQMP54Y6VNETR5TBQWQ5WOYLQ7O2VQPTRNGD74Q5Q4HDZFYPMA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:15.387535187 (35h36m51.427108317s) [noise=true , address=77.90.31.10:4517 , piece_id=Z3KQUV23D72RCUTWQNI23PPLZ3FX3AL5RA7SKWJSSGCQYUAT5KJQ, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:16.537381878 (35h36m52.576954998s) [noise=true , address=45.91.116.13:4153 , piece_id=VLRPMW2DLA55XJ5O63NSURH6HOPAREMCNJMQ5FPZY3FSLILBOFPA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:17.070063103 (35h36m53.109636223s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZCCXZMDSLCLWFY2XTKLJAP4ZDTP6WTHGLGUSVPISLJXVUNPZDXA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: <nil> Sep 8 01:05:42.069703118 (35h37m18.109276238s) [noise=true , address=145.79.227.13:4804 , piece_id=5DGX3H5EAZBOWFDERDVBZ4HDONUIUZBQYQBVP4BMOI7G2ILAVDBA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:33168->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.069918490 (35h37m18.109491611s) [noise=true , address=145.79.227.11:4729 , piece_id=KVPLH34LRLZEBDZ5JUGE7EDEY2Z6N6PWCOADOLGIBMKCUEXQBUHA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070009705 (35h37m18.109582825s) [noise=true , address=145.79.227.7:4293 , piece_id=NMDQ53GRSUXWJDWE4BXZGSJISBSH2HN267IZNGMOUVMRF5QG6N5A, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: read tcp 188.166.225.170:50614->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070048449 (35h37m18.109621569s) [noise=true , address=145.79.227.6:4259 , piece_id=OIYFCH2NOOROE6BKFEI2YNM72U6ZJCNMFP3HV6RZTTSXS2Z5KLCA, seg_pos={0 0}, upload=31297]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:49928->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 8 01:05:42.070099257 (35h37m18.109672377s) [seg_pos={0 0}, upload=31297]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 8 01:05:42.335470264 (35h37m18.375043394s) [upload=31297]: Done uploading segments... Sep 8 01:05:42.335501313 (35h37m18.375074433s) [upload=31297]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.2677198s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.151050805s FAIL (2025-09-07 20:48:12 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.151107022s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.007045167s FAIL (2025-09-07 20:47:43 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.007076197s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.341812564s FAIL (2025-09-07 20:46:45 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.341842181s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.367190696s FAIL (2025-09-07 20:46:13 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.367213209s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.143322247s FAIL (2025-09-07 20:45:58 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.143351513s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.870609255s FAIL (2025-09-07 20:45:21 UTC) orbiter warning: context deadline exceeded Duration: 5.870631416s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.093664652s FAIL (2025-09-07 20:44:45 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.093719027s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.840123597s FAIL (2025-09-07 20:44:29 UTC) orbiter warning: context deadline exceeded Duration: 5.840144708s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.176899081s FAIL (2025-09-07 20:43:35 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.176921413s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.718112919s FAIL (2025-09-07 20:43:27 UTC) orbiter warning: context deadline exceeded Duration: 5.71813455s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.167075634s FAIL (2025-09-07 20:42:33 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.167101312s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.765467992s FAIL (2025-09-07 20:42:33 UTC) orbiter warning: context deadline exceeded Duration: 5.765509091s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)