Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X.......X..................X.................] duration: 7.208347039s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:49:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.208381043s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X.......X..................X................] duration: 9.172621217s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:48:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 9.172648089s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X.......X..................X...............] duration: 7.760767995s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:47:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.760800087s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X.......X..................X..............] duration: 8.910515954s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:47:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.910546043s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X.......X..................X.............] duration: 8.888887362s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:46:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.888921317s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X.......X..................X............] duration: 7.520532727s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:45:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.520563947s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X.......X..................X...........] duration: 7.983729603s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:44:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.983761754s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X.......X..................X..........] duration: 8.089235331s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:43:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.089263915s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X.......X..................X.........] duration: 8.21490908s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:43:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.214931252s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................X.......X..................X........] duration: 6.514013535s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:41:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.514045153s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................X.......X..................X.......] duration: 6.816180836s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:40:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.816210523s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................X.......X..................X......] duration: 7.112085008s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:40:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.112115056s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................X.......X..................X.....] duration: 7.537042722s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:39:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.537074122s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................X.......X..................X....] duration: 7.324953974s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:39:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.324990924s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................X.......X..................X...] duration: 6.609847865s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:38:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.609878974s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................X.......X..................X..] duration: 11.190905511s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:37:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 11.190943745s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................X.......X..................X.] duration: 10.899057936s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:36:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 10.899090437s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................X.......X..................X] duration: 30.254808694s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:36:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.254839974s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.......X..................] duration: 7.971743618s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:35:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.971781972s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X.......X.................] duration: 8.472845848s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:33:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 8.472880053s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................X.......X................] duration: 8.00810106s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:32:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 8.008132068s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.......X...............] duration: 7.805285253s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:32:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.805320711s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.......X..............] duration: 7.655990228s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:31:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.656014835s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X.......X.............] duration: 7.561038684s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:30:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.561069572s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X.......X............] duration: 6.688812933s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:29:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 6.688836528s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X.......X...........] duration: 6.903975912s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:28:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 6.903998174s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X.......X..........] duration: 8.019933748s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:27:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 8.01997193s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 322.698057ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/7fa61a589e9879b2 FAIL (2025-11-11 15:26:54 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38942->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 322.727094ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X.......X.........] duration: 7.832316353s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:26:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.832333936s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X.......X........] duration: 6.617024711s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:26:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 6.617047795s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 337.992231ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/7fa61a589e9879b2 FAIL (2025-11-11 15:25:49 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38942->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 338.022018ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................X.......X.......] duration: 7.037755556s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:25:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.037790602s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 482.88039ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/7fa61a589e9879b2 FAIL (2025-11-11 15:25:07 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38942->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 482.924955ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 302.988717ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/7fa61a589e9879b2 FAIL (2025-11-11 15:24:39 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38942->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 303.019105ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 577.71643ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/7fa61a589e9879b2 FAIL (2025-11-11 15:24:16 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38942->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 577.746758ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X.......X......] duration: 7.683243755s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:24:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.683337773s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X.......X.....] duration: 15.06710473s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:23:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 15.067222494s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X.......X....] duration: 7.476153902s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:23:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.476173719s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 321.142916ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/7fa61a589e9879b2 FAIL (2025-11-11 15:23:07 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38942->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 321.172313ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................X.......X...] duration: 7.405056056s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:22:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.405087996s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 297.645261ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/7fa61a589e9879b2 FAIL (2025-11-11 15:22:34 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38942->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 297.660209ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 332.861854ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/7fa61a589e9879b2 FAIL (2025-11-11 15:21:50 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38942->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 332.889967ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X.......X..] duration: 7.659316198s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:21:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.659344614s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 321.484929ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/7fa61a589e9879b2 FAIL (2025-11-11 15:21:12 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38942->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 321.513663ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X.......X.] duration: 7.776849795s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:20:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.776876495s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 549.018131ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/7fa61a589e9879b2 FAIL (2025-11-11 15:19:57 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38942->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 549.044603ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X.......X] duration: 30.251368919s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:19:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 30.251394146s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 336.740396ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/7fa61a589e9879b2 FAIL (2025-11-11 15:19:40 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38942->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 336.771686ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 328.772985ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/7fa61a589e9879b2 FAIL (2025-11-11 15:18:52 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38942->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 328.804926ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 8.588962275s trace: https://jaeger.dev.storj.io/trace/24f188b305832b39 FAIL (2025-11-11 15:18:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 04 (55h8m30.690737755s) [upload=48590]: Done starting upload Nov 11 15:12:32.139311849 (55h8m30.69078231s) [upload=48590]: Uploading segments... Nov 11 15:12:32.139474188 (55h8m30.690944639s) [upload=48590]: Got next segment. Inline: false Nov 11 15:12:32.139518191 (55h8m30.690988642s) [upload=48590]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 15:12:32.139524143 (55h8m30.690994594s) [upload=48590]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 15:12:32.407160882 (55h8m30.958631334s) [seg_pos={0 0}, upload=48590]: Begin upload segment... Nov 11 15:12:32.407209555 (55h8m30.958680016s) [seg_pos={0 0}, upload=48590]: Done begin upload segment. Nov 11 15:12:32.407237057 (55h8m30.958707518s) [noise=true , address=145.79.227.3:4150 , piece_id=QGP2M23UQC43AYM4MJY3EPU26VQLQUU4U2UNTZLTVJK2UT4DUUPQ, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:32.409777184 (55h8m30.961247645s) [noise=true , address=145.79.227.13:4804 , piece_id=BDBGYNNMESM5FFAC6ZJL2KI4FX7V4KC3JT3MV5RZAMSIBGSMFVHQ, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:32.410920572 (55h8m30.962391033s) [noise=true , address=145.79.227.21:4473 , piece_id=K2QELFCJEZFCUKSJXLB6XAZ4MWB7KEKI2RQXBLNWM2GXFYTDTGXQ, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:32.411921438 (55h8m30.963391889s) [noise=true , address=145.79.227.26:4654 , piece_id=NI77RPURJFK6FR6GSLOPSMERGV6P4ENHEUAEYLDM2NWVAVL7ADVQ, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:32.412637050 (55h8m30.964107511s) [noise=true , address=145.79.227.19:4079 , piece_id=N2OLSJFSCU2SHGREFJQRVNAYL253HW6RSLFCQQH73XSD2X6RSIQQ, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:32.413456450 (55h8m30.964926912s) [noise=true , address=145.79.227.20:4438 , piece_id=GQZVHP6W66FHBXX4CZSHH7XSXSUBYGPJ6YG6PXIVRXP3OYDZ3M2A, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:32.414667867 (55h8m30.966138328s) [noise=true , address=145.79.227.28:4725 , piece_id=5GE744ZK2RANSEVQQCLDXAUOU4BCJU7QKA7QKUGTV7YE3K6BJNUA, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:32.415580074 (55h8m30.967050535s) [noise=true , address=145.79.227.17:4970 , piece_id=NCNNN33LUOZSLMKIXB5HYSSDVX4OQET2ZHO4FXJ7URUGQKQ4FAOQ, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:32.416440363 (55h8m30.967910824s) [noise=true , address=45.91.116.5:4297 , piece_id=56E5AAFLMVQQRZ4QRNFIZGPIP3D7SM4DK2AX7BDTEPCDN55LR36Q, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:32.417231970 (55h8m30.968702431s) [noise=true , address=145.79.227.15:4873 , piece_id=7RT5FDC4EAE4WZ5TCYEXF53TK2JOCVKW7TQAA2FMKAMDGCOFYEJA, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:32.418328648 (55h8m30.969799109s) [noise=true , address=77.90.31.16:4734 , piece_id=MLZZ7YMCAQ7X7AHLA3TS5EQAF3MYP2MN52VSW34CJ6UOB624LBYQ, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:32.422014076 (55h8m30.973484557s) [noise=true , address=145.79.227.9:4367 , piece_id=QRSC6XLUQ6XDB3R67PG4IBYW4U4AEZJ7CQFY4BEDQI3GUEORM6WA, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:32.422938577 (55h8m30.974409038s) [noise=true , address=145.79.227.27:4691 , piece_id=ZABAKMXZC2HSYM7EHAKRVJWEVMGB2VG5WDUI6HNXXIET23KSNGOA, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:32.428843361 (55h8m30.980313822s) [noise=true , address=45.91.116.4:4939 , piece_id=2MANI2JLKD4FIDOD6FWRRMAZGRO2C3COKKIYZUZ6RT32UKNGK4DA, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:32.429122432 (55h8m30.980592893s) [noise=true , address=77.90.31.11:4554 , piece_id=7VFKHHJVK6AWKB36P4EFQHVRRWPUETYN6HT2XE7IQUIYN5S55BRQ, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:32.432451390 (55h8m30.983921841s) [noise=true , address=145.79.227.31:4798 , piece_id=FFDDNIHBQGGKVDQD33OULOFJ2GZ6BTR3SCC426PVD737T6MPLCQA, seg_pos={0 0}, upload=48590]: Uploading piece... Nov 11 15:12:34.249518680 (55h8m32.800989141s) [upload=48590]: Next returned nil segment Nov 11 15:12:34.249535282 (55h8m32.801005733s) [upload=48590]: Waiting for error group managing segments... Nov 11 15:12:34.549069771 (55h8m33.100540233s) [noise=true , address=45.91.116.4:4939 , piece_id=2MANI2JLKD4FIDOD6FWRRMAZGRO2C3COKKIYZUZ6RT32UKNGK4DA, seg_pos={0 0}, upload=48590]: Done uploading piece. err: <nil> Nov 11 15:12:34.634728415 (55h8m33.186198876s) [noise=true , address=77.90.31.16:4734 , piece_id=MLZZ7YMCAQ7X7AHLA3TS5EQAF3MYP2MN52VSW34CJ6UOB624LBYQ, seg_pos={0 0}, upload=48590]: Done uploading piece. err: <nil> Nov 11 15:12:34.648503204 (55h8m33.199973666s) [noise=true , address=45.91.116.5:4297 , piece_id=56E5AAFLMVQQRZ4QRNFIZGPIP3D7SM4DK2AX7BDTEPCDN55LR36Q, seg_pos={0 0}, upload=48590]: Done uploading piece. err: <nil> Nov 11 15:12:34.686662118 (55h8m33.238132579s) [noise=true , address=77.90.31.11:4554 , piece_id=7VFKHHJVK6AWKB36P4EFQHVRRWPUETYN6HT2XE7IQUIYN5S55BRQ, seg_pos={0 0}, upload=48590]: Done uploading piece. err: <nil> Nov 11 15:12:52.012489181 (55h8m50.563959642s) [noise=true , address=145.79.227.9:4367 , piece_id=QRSC6XLUQ6XDB3R67PG4IBYW4U4AEZJ7CQFY4BEDQI3GUEORM6WA, seg_pos={0 0}, upload=48590]: Done uploading piece. err: <nil> Nov 11 15:12:53.163094636 (55h8m51.714565097s) [noise=true , address=145.79.227.21:4473 , piece_id=K2QELFCJEZFCUKSJXLB6XAZ4MWB7KEKI2RQXBLNWM2GXFYTDTGXQ, seg_pos={0 0}, upload=48590]: Done uploading piece. err: <nil> Nov 11 15:12:54.005113607 (55h8m52.556584058s) [noise=true , address=145.79.227.28:4725 , piece_id=5GE744ZK2RANSEVQQCLDXAUOU4BCJU7QKA7QKUGTV7YE3K6BJNUA, seg_pos={0 0}, upload=48590]: Done uploading piece. err: <nil> Nov 11 15:12:55.792527625 (55h8m54.343998086s) [noise=true , address=145.79.227.19:4079 , piece_id=N2OLSJFSCU2SHGREFJQRVNAYL253HW6RSLFCQQH73XSD2X6RSIQQ, seg_pos={0 0}, upload=48590]: Done uploading piece. err: <nil> Nov 11 15:12:56.429130256 (55h8m54.980600717s) [noise=true , address=145.79.227.13:4804 , piece_id=BDBGYNNMESM5FFAC6ZJL2KI4FX7V4KC3JT3MV5RZAMSIBGSMFVHQ, seg_pos={0 0}, upload=48590]: Done uploading piece. err: <nil> Nov 11 15:12:57.754795865 (55h8m56.306266317s) [noise=true , address=145.79.227.20:4438 , piece_id=GQZVHP6W66FHBXX4CZSHH7XSXSUBYGPJ6YG6PXIVRXP3OYDZ3M2A, seg_pos={0 0}, upload=48590]: Done uploading piece. err: <nil> Nov 11 15:12:57.776570418 (55h8m56.32804087s) [noise=true , address=145.79.227.31:4798 , piece_id=FFDDNIHBQGGKVDQD33OULOFJ2GZ6BTR3SCC426PVD737T6MPLCQA, seg_pos={0 0}, upload=48590]: Done uploading piece. err: <nil> Nov 11 15:12:58.031684232 (55h8m56.583154683s) [noise=true , address=145.79.227.17:4970 , piece_id=NCNNN33LUOZSLMKIXB5HYSSDVX4OQET2ZHO4FXJ7URUGQKQ4FAOQ, seg_pos={0 0}, upload=48590]: Done uploading piece. err: <nil> Nov 11 15:13:00.746621531 (55h8m59.298091992s) [noise=true , address=145.79.227.15:4873 , piece_id=7RT5FDC4EAE4WZ5TCYEXF53TK2JOCVKW7TQAA2FMKAMDGCOFYEJA, seg_pos={0 0}, upload=48590]: Done uploading piece. err: <nil> Nov 11 15:13:00.746642571 (55h8m59.298113032s) [seg_pos={0 0}, upload=48590]: Segment reached optimal threshold of 13 pieces. Nov 11 15:13:00.746867810 (55h8m59.298338261s) [noise=true , address=145.79.227.3:4150 , piece_id=QGP2M23UQC43AYM4MJY3EPU26VQLQUU4U2UNTZLTVJK2UT4DUUPQ, seg_pos={0 0}, upload=48590]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: write tcp 188.166.225.170:38954->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:38954->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38954->145.79.227.3:4150: use of closed network connection Nov 11 15:13:00.746877799 (55h8m59.29834825s) [noise=true , address=145.79.227.3:4150 , piece_id=QGP2M23UQC43AYM4MJY3EPU26VQLQUU4U2UNTZLTVJK2UT4DUUPQ, seg_pos={0 0}, upload=48590]: Long tail cancellation due to optimal threshold Nov 11 15:13:00.747210924 (55h8m59.298681385s) [noise=true , address=145.79.227.26:4654 , piece_id=NI77RPURJFK6FR6GSLOPSMERGV6P4ENHEUAEYLDM2NWVAVL7ADVQ, seg_pos={0 0}, upload=48590]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah): protocol: expected piece hash; context canceled; EOF Nov 11 15:13:00.747220772 (55h8m59.298691223s) [noise=true , address=145.79.227.26:4654 , piece_id=NI77RPURJFK6FR6GSLOPSMERGV6P4ENHEUAEYLDM2NWVAVL7ADVQ, seg_pos={0 0}, upload=48590]: Long tail cancellation due to optimal threshold Nov 11 15:13:00.747353104 (55h8m59.298823555s) [noise=true , address=145.79.227.27:4691 , piece_id=ZABAKMXZC2HSYM7EHAKRVJWEVMGB2VG5WDUI6HNXXIET23KSNGOA, seg_pos={0 0}, upload=48590]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg): protocol: write tcp 188.166.225.170:58754->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:58754->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58754->145.79.227.27:4691: use of closed network connection Nov 11 15:13:00.747360348 (55h8m59.298830799s) [noise=true , address=145.79.227.27:4691 , piece_id=ZABAKMXZC2HSYM7EHAKRVJWEVMGB2VG5WDUI6HNXXIET23KSNGOA, seg_pos={0 0}, upload=48590]: Long tail cancellation due to optimal threshold Nov 11 15:13:00.747376890 (55h8m59.298847351s) [seg_pos={0 0}, upload=48590]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 15:13:00.747505805 (55h8m59.298976276s) [upload=48590]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 15:13:00.747515083 (55h8m59.298985534s) [upload=48590]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 15:13:01.055107507 (55h8m59.606577968s) [upload=48590]: Done uploading segments... Nov 11 15:13:01.055144638 (55h8m59.606615099s) [upload=48590]: Upload finished. err: <nil> Duration: 8.588999367s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)