Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.78484043s FAIL (2025-07-03 12:40:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.784864516s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.733846328s FAIL (2025-07-03 12:39:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.733869232s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.580015439s FAIL (2025-07-03 12:38:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.580034094s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.649944539s FAIL (2025-07-03 12:37:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.649974546s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.730272836s FAIL (2025-07-03 12:36:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.730298044s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.787890352s FAIL (2025-07-03 12:35:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.787920669s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.702864568s FAIL (2025-07-03 12:35:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.702905116s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.693380876s FAIL (2025-07-03 12:34:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.693405913s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.696756639s FAIL (2025-07-03 12:33:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.69678858s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.755434803s FAIL (2025-07-03 12:32:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.755458628s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.674237165s FAIL (2025-07-03 12:32:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.674266571s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.644089634s FAIL (2025-07-03 12:31:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.644119511s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.709732542s FAIL (2025-07-03 12:31:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.709775044s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.91997801s FAIL (2025-07-03 12:30:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.920004793s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.732213034s FAIL (2025-07-03 12:30:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.732240857s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.637909014s FAIL (2025-07-03 12:29:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.637933962s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.802071917s FAIL (2025-07-03 12:29:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.802093567s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.66011652s FAIL (2025-07-03 12:28:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.660143041s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.727172882s FAIL (2025-07-03 12:27:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.727207299s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.647822889s FAIL (2025-07-03 12:27:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.647850711s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.78121377s FAIL (2025-07-03 12:26:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.78123497s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.689484271s FAIL (2025-07-03 12:26:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.689516483s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.635663577s FAIL (2025-07-03 12:25:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.635693785s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.709967644s FAIL (2025-07-03 12:24:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.709993513s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.642507629s FAIL (2025-07-03 12:23:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.642547837s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.684368506s FAIL (2025-07-03 12:22:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.684393484s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.910259138s FAIL (2025-07-03 12:22:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.910289555s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.249927606s FAIL (2025-07-03 12:21:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12:20:30.340466822 (243h10m5.111108157s) [upload=194445]: Done uploading segments... Jul 3 12:20:30.340488313 (243h10m5.111129648s) [upload=194445]: Upload finished. err: <nil> Jul 3 12:21:01.408651148 (243h10m36.179292483s) [upload=194453]: Starting upload Jul 3 12:21:01.408712885 (243h10m36.17935422s) [upload=194453]: Done starting upload Jul 3 12:21:01.408844107 (243h10m36.179485442s) [upload=194453]: Uploading segments... Jul 3 12:21:01.408952825 (243h10m36.17959416s) [upload=194453]: Got next segment. Inline: false Jul 3 12:21:01.409001809 (243h10m36.179643144s) [upload=194453]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 3 12:21:01.409007830 (243h10m36.179649165s) [upload=194453]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 3 12:21:01.644214556 (243h10m36.414855891s) [seg_pos={0 0}, upload=194453]: Begin upload segment... Jul 3 12:21:01.644275844 (243h10m36.414917179s) [seg_pos={0 0}, upload=194453]: Done begin upload segment. Jul 3 12:21:01.644337802 (243h10m36.414979147s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644386506 (243h10m36.415027841s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644907011 (243h10m36.415548356s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.644973589 (243h10m36.415614924s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645307067 (243h10m36.415948412s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645487481 (243h10m36.416128836s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.645723062 (243h10m36.416364407s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.646132456 (243h10m36.416773791s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647101249 (243h10m36.417742594s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647478560 (243h10m36.418119905s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.647867955 (243h10m36.41850929s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648197536 (243h10m36.418838871s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648581791 (243h10m36.419223126s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.648922433 (243h10m36.419563768s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649263816 (243h10m36.419905151s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:01.649599538 (243h10m36.420240873s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Uploading piece... Jul 3 12:21:03.277210623 (243h10m38.047851948s) [upload=194453]: Next returned nil segment Jul 3 12:21:03.277228466 (243h10m38.047869812s) [upload=194453]: Waiting for error group managing segments... Jul 3 12:21:03.538906484 (243h10m38.309547809s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQWV7C3ZT6ZPQEJURAMMEL3G4WPHHQZ5BEH7TUMIWS2HM3UPRNA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.595541314 (243h10m38.366182649s) [noise=true , address=104.36.82.10:4079 , piece_id=MWE5X5BSHR2V3BRKF45NUHHXVTHRIWNSYPXFXM2EALD22T32D2PQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.599492944 (243h10m38.370134269s) [noise=true , address=104.36.82.6:4875 , piece_id=LKDAWY4GCEOO4Q3WO2ASWB3443IDZ5TAUSDVQ4OWLU7XUOYT2D3A, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.613103161 (243h10m38.383744496s) [noise=true , address=104.36.82.11:4115 , piece_id=ECQAOD7WN7Z5QKSC4U43N3MRVDX5V4N2AVV23DGVPCFXE7ZMJYGA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.642864218 (243h10m38.413505553s) [noise=true , address=104.36.82.15:4257 , piece_id=ELFIRLJUWLKAGFRP5YLKKG2ZN2WVEH7YFT3A5HEQAYZJTYQ3A7CA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.735480123 (243h10m38.506121458s) [noise=true , address=104.36.82.14:4223 , piece_id=4YBLMZXCCDVJYVLFXKLPOMUSNRXCIHOQGVTRQ5AGSOWVISFW7SMA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.756306776 (243h10m38.526948151s) [noise=true , address=104.36.82.9:4006 , piece_id=IYOHU4CL6ILDJFSTDLQRGGNOWSKUFTENWL3GORBQBII4ZLHNBE7Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:03.855913565 (243h10m38.6265549s) [noise=true , address=104.36.82.2:4732 , piece_id=UDJSISRGTNLT4GQ3ARF6T5WVIWWLZUZMYI7HRKCE4356LC73C7IA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:05.910841421 (243h10m40.681482756s) [noise=true , address=104.36.82.5:4840 , piece_id=NIJ4LKGQXTHHXMA6ALFDY6CZM3YPP5NM2FKWAXSD2BY5OOIJCTDQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.150146516 (243h10m40.920787851s) [noise=true , address=45.91.116.11:4119 , piece_id=PI6O3OLWQHU2BGSCOYU67SMD7IKKYXE4L7XI6F7SYIAER4WRCM4Q, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:06.291707347 (243h10m41.062348682s) [noise=true , address=45.91.116.5:4298 , piece_id=HMJIZJ6PFXER7IK6ZRDSACIQZUC6IUYET45BAQVQTNKFCTF44TLQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:13.374083231 (243h10m48.144724566s) [noise=true , address=45.91.116.7:4011 , piece_id=2ZZQFQPPPVJ4OXW2JASGJXTLVYG6MSQLBPKZUEDMQZQUHOFEHDIQ, seg_pos={0 0}, upload=194453]: Done uploading piece. err: <nil> Jul 3 12:21:30.708319515 (243h11m5.47896085s) [noise=true , address=77.90.31.10:4517 , piece_id=O7Y6N7VPKZK5A36NJAUEEZZ3VF7FKRE7I6RSQXU7BKETX574CERA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47600->77.90.31.10:4517: use of closed network connection Jul 3 12:21:30.708379560 (243h11m5.479020895s) [noise=true , address=77.90.31.15:4697 , piece_id=BEIUG2I7GQOEMEXNK7JU3VNFK2FT5ITLDCBEDIN4RKD7B52JCGFA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39210->77.90.31.15:4697: use of closed network connection Jul 3 12:21:30.708614900 (243h11m5.479256235s) [noise=true , address=77.90.31.12:4592 , piece_id=V43L4RUKOSVEBAD2YPARQXGRHY77OE4F5DECHDSMR2TZQZYTANEA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58504->77.90.31.12:4592: use of closed network connection Jul 3 12:21:30.709001149 (243h11m5.479642484s) [noise=true , address=77.90.31.14:4663 , piece_id=EHMWGSCRLVQKN5J6QK5DS5NBELDW7BCDBOSZ564WM2FMDWVT56EA, seg_pos={0 0}, upload=194453]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53534->77.90.31.14:4663: use of closed network connection Jul 3 12:21:30.709030845 (243h11m5.479672181s) [seg_pos={0 0}, upload=194453]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 3 12:21:30.956154170 (243h11m5.726795505s) [upload=194453]: Done uploading segments... Jul 3 12:21:30.956179397 (243h11m5.726820732s) [upload=194453]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.249953014s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.871400425s FAIL (2025-07-03 09:26:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.871425362s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.772587228s FAIL (2025-07-03 09:26:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.772611284s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.743958065s FAIL (2025-07-03 09:25:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.743977011s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 7.278520693s FAIL (2025-07-03 09:24:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 7.278545191s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 8.727760108s FAIL (2025-07-03 09:23:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 8.72779284s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.696907025s FAIL (2025-07-03 09:22:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.696932665s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.648219989s FAIL (2025-07-03 09:21:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.648244677s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.677924691s FAIL (2025-07-03 09:20:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.677951111s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.137662473s FAIL (2025-07-03 09:19:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 6.137690476s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.656109513s FAIL (2025-07-03 09:18:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.656137357s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.618419877s FAIL (2025-07-03 09:17:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.618447732s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.885921592s FAIL (2025-07-03 09:16:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.885946238s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.701761822s FAIL (2025-07-03 09:15:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.701788793s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.605276524s FAIL (2025-07-03 09:14:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.605301421s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.761849856s FAIL (2025-07-03 09:14:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.761876077s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.673977297s FAIL (2025-07-03 09:13:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.674001302s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.840204732s FAIL (2025-07-03 09:12:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.840231162s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.695324407s FAIL (2025-07-03 09:11:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.695351078s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.3964418s FAIL (2025-07-03 09:11:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 6.396471816s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.666160785s FAIL (2025-07-03 09:10:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.666189721s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.827768117s FAIL (2025-07-03 09:09:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.827796432s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.83880022s FAIL (2025-07-03 09:08:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== eg_pos={0 0}, upload=191557]: Done begin upload segment. Jul 3 08:42:15.207606115 (239h31m49.97824745s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207617276 (239h31m49.978258611s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.207939112 (239h31m49.978580437s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208127453 (239h31m49.978768788s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208408060 (239h31m49.979049395s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208733232 (239h31m49.979374567s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.208762568 (239h31m49.979403903s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209000163 (239h31m49.979641498s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209190928 (239h31m49.979832263s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209225824 (239h31m49.979867159s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209458689 (239h31m49.980100025s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209620710 (239h31m49.980262045s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209709409 (239h31m49.980350744s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.209930712 (239h31m49.980572047s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210133471 (239h31m49.980774796s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:15.210514820 (239h31m49.981156155s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Uploading piece... Jul 3 08:42:16.882521289 (239h31m51.653162624s) [upload=191557]: Next returned nil segment Jul 3 08:42:16.882541407 (239h31m51.653182742s) [upload=191557]: Waiting for error group managing segments... Jul 3 08:42:17.149026216 (239h31m51.919667541s) [noise=true , address=104.36.82.9:4005 , piece_id=KSFAUN6CIZYZZFTEX7T6YP43GH2K3ZILBZIYCYOEIJ5ILG6EQHJA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.190174868 (239h31m51.960816203s) [noise=true , address=104.36.82.15:4257 , piece_id=2HUFWH2NSHUB3MHQ52PO3NCGWIVJ4CIJJD24DBTXMLB67BSZ3SOQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.248711316 (239h31m52.019352651s) [noise=true , address=104.36.82.7:4910 , piece_id=OXXAAXERXA7HTYIMCTPB3GD3FFUCQBOUI7RVOCHXJ2UMKDILIGJQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:17.278374525 (239h31m52.04901586s) [noise=true , address=104.36.82.5:4838 , piece_id=T3RDNEMK5FR33WRBRLDO2UT34SLOUFBKLW652TB6QC3CHNKXJDPA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:18.807962921 (239h31m53.578604256s) [noise=true , address=45.91.116.11:4117 , piece_id=5LVVNVUU2TIE7CNXTTW7DTTSEW357KESUJZV6KI7ME2FGYFGSYVQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:19.321679918 (239h31m54.092321253s) [noise=true , address=45.91.116.9:4263 , piece_id=V5NDZMYAFF3EJEOEHVOGSOHE5O566MQ7MVMQRHS2UGPHU2DEDFKA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.063591957 (239h31m54.834233292s) [noise=true , address=45.91.116.8:4048 , piece_id=QMWQ7RAR6WEQKF22D36JGA42X77Z77JXY45OK3W62O23NGBEOCTQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:20.123633384 (239h31m54.89427478s) [noise=true , address=45.91.116.12:4227 , piece_id=RMA77AGKLIGQRKA75UWNJZVJRG4PM3YAQ2TD2HHF3DJNYHDWZJLA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:23.485425564 (239h31m58.256066899s) [noise=true , address=45.91.116.4:4937 , piece_id=XPTX4FSUYKVYZFARBCNOJ7ZQUSAM6DYMEXAW7KAJEWRIJX36NIFQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: <nil> Jul 3 08:42:44.322062124 (239h32m19.092703459s) [noise=true , address=77.90.31.18:4805 , piece_id=DT6EIRTBDBPYYCBKALBDGPP75WMMU24WGCT25MSW5V7SDY2PQJAQ, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44746->77.90.31.18:4805: use of closed network connection Jul 3 08:42:44.322162176 (239h32m19.092803511s) [noise=true , address=77.90.31.17:4771 , piece_id=OEZ3QX7CRFXL73EXECTKAGUC2PX3VUHKS3GCWFKU3IOIOLGPECQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58328->77.90.31.17:4771: use of closed network connection Jul 3 08:42:44.322230095 (239h32m19.092871431s) [noise=true , address=77.90.31.21:4879 , piece_id=MGCCS4G7PDI25GDF6BG3WDMQUKQH4D6YXBLFHDO4SA53VMATFB5Q, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57688->77.90.31.21:4879: use of closed network connection Jul 3 08:42:44.322270814 (239h32m19.092912149s) [noise=true , address=77.90.31.13:4626 , piece_id=RZWLWSJSGH73FXB5ANNCZZGS3KOWE3AI376KUZGKFU4QYVRC27MA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45144->77.90.31.13:4626: use of closed network connection Jul 3 08:42:44.322307303 (239h32m19.092948638s) [noise=true , address=77.90.31.10:4518 , piece_id=ZVPNJ7CE6RFAVCXJGW6PBB3DBMFA7ZHLRYYSPON4JASHWYL44GIA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34754->77.90.31.10:4518: use of closed network connection Jul 3 08:42:44.322362769 (239h32m19.093004105s) [noise=true , address=77.90.31.14:4662 , piece_id=BPVTIZXTAH36DTFA2MHNLKGQ7CRZUF2WAOV73AV7LMJEO6C2JHQA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 188.166.225.170:40830->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322447972 (239h32m19.093089338s) [noise=true , address=77.90.31.19:4841 , piece_id=G2VXPN4FGNTXPZ522YAUGG7FKZFKSTHTZ4CNN2FYHDDHILITLBYA, seg_pos={0 0}, upload=191557]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:50222->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 3 08:42:44.322490354 (239h32m19.093131689s) [seg_pos={0 0}, upload=191557]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 3 08:42:44.562459395 (239h32m19.33310073s) [upload=191557]: Done uploading segments... Jul 3 08:42:44.562485345 (239h32m19.33312668s) [upload=191557]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.838839656s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)