Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X.X..............................................] duration: 5.606007894s FAIL (2025-06-25 13:38:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.606032091s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X.X.............................................] duration: 5.731571111s FAIL (2025-06-25 13:37:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.731594207s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X.X............................................] duration: 5.683888323s FAIL (2025-06-25 13:37:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.683924282s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X.X...........................................] duration: 5.672960784s FAIL (2025-06-25 13:36:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.672985583s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X.X..........................................] duration: 5.715488863s FAIL (2025-06-25 13:36:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.715517518s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X.X.........................................] duration: 5.742716811s FAIL (2025-06-25 13:34:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.742746138s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X.X........................................] duration: 5.641389304s FAIL (2025-06-25 13:33:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.641469058s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................X.X.......................................] duration: 5.918590032s FAIL (2025-06-25 13:32:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.918611705s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................X.X......................................] duration: 5.656265475s FAIL (2025-06-25 13:31:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.656290504s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................X.X.....................................] duration: 5.66803893s FAIL (2025-06-25 13:31:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.668070592s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................X.X....................................] duration: 5.835688731s FAIL (2025-06-25 13:30:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.835713298s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................X.X...................................] duration: 5.663540933s FAIL (2025-06-25 13:29:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.663560953s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................X.X..................................] duration: 5.667601789s FAIL (2025-06-25 13:28:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.667625015s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................X.X.................................] duration: 5.74550035s FAIL (2025-06-25 13:27:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.745525299s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................X.X................................] duration: 5.869879714s FAIL (2025-06-25 13:26:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.869905655s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................X.X...............................] duration: 5.834429491s FAIL (2025-06-25 13:25:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.834456924s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................X.X..............................] duration: 5.733329748s FAIL (2025-06-25 13:24:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.733355437s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................X.X.............................] duration: 5.777900665s FAIL (2025-06-25 13:24:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.777924933s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................X.X............................] duration: 5.789394307s FAIL (2025-06-25 13:22:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.789421149s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................X.X...........................] duration: 5.695771093s FAIL (2025-06-25 13:21:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.695790562s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................X.X..........................] duration: 5.713248316s FAIL (2025-06-25 13:21:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.713276541s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................X.X.........................] duration: 5.62675563s FAIL (2025-06-25 13:19:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.626798854s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.X........................] duration: 5.651738425s FAIL (2025-06-25 13:18:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.651758685s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X.X.......................] duration: 5.699889229s FAIL (2025-06-25 13:18:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.699909737s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................X.X......................] duration: 5.688684581s FAIL (2025-06-25 13:17:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.688725941s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.X.....................] duration: 5.626331602s FAIL (2025-06-25 13:16:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.626365698s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.X....................] duration: 5.861518759s FAIL (2025-06-25 13:15:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.861544397s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X.X...................] duration: 5.661008527s FAIL (2025-06-25 13:15:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.661047211s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X.X..................] duration: 5.696337872s FAIL (2025-06-25 13:14:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.696365205s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X.X.................] duration: 5.708078098s FAIL (2025-06-25 13:14:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.708102555s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X.X................] duration: 5.603045941s FAIL (2025-06-25 13:13:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.603070849s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X.X...............] duration: 5.761644504s FAIL (2025-06-25 13:12:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.761671286s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X.X..............] duration: 5.793082512s FAIL (2025-06-25 13:12:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.793108252s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................X.X.............] duration: 5.64985624s FAIL (2025-06-25 13:11:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.649899545s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X.X............] duration: 5.731108455s FAIL (2025-06-25 13:10:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.731132681s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X.X...........] duration: 5.754725592s FAIL (2025-06-25 13:09:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.754746522s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X.X..........] duration: 5.795732022s FAIL (2025-06-25 13:08:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.795753956s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................X.X.........] duration: 5.617440997s FAIL (2025-06-25 13:08:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.617462319s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X.X........] duration: 5.664818048s FAIL (2025-06-25 13:07:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.664848916s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X.X.......] duration: 5.615618533s FAIL (2025-06-25 13:07:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.615652098s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X.X......] duration: 5.916464939s FAIL (2025-06-25 13:06:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.916489857s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X.X.....] duration: 5.638320189s FAIL (2025-06-25 13:05:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.638359395s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X.X....] duration: 5.714489216s FAIL (2025-06-25 13:04:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.71451197s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................X.X...] duration: 5.624186026s FAIL (2025-06-25 13:03:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.624207258s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................X.X..] duration: 5.810636765s FAIL (2025-06-25 13:02:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 5.810667384s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................................X.X.] duration: 17.492244759s FAIL (2025-06-25 13:02:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 17.49226005s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................X.X] duration: 30.234149075s FAIL (2025-06-25 13:01:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== e_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941388971 (51h50m56.712030306s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941623656 (51h50m56.712264991s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.941854513 (51h50m56.712495848s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942064570 (51h50m56.712705905s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942282010 (51h50m56.712923335s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942496205 (51h50m56.71313754s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942703717 (51h50m56.713345052s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.942880470 (51h50m56.713521805s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943056550 (51h50m56.713697885s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943230217 (51h50m56.713871552s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943408452 (51h50m56.714049787s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943621615 (51h50m56.71426295s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:21.943804579 (51h50m56.714445914s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Uploading piece... Jun 25 13:01:23.625961904 (51h50m58.396603239s) [upload=41394]: Next returned nil segment Jun 25 13:01:23.625993946 (51h50m58.396635281s) [upload=41394]: Waiting for error group managing segments... Jun 25 13:01:23.927659351 (51h50m58.698300676s) [noise=true , address=104.36.82.7:4912 , piece_id=4PWENM3XQ3OC3FQXXOYOHJL4OCUWM5XHHG5FSYE2INIUTOQJH4JA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:23.945179161 (51h50m58.715820496s) [noise=true , address=104.36.82.3:4765 , piece_id=FRLDDLMN46BOORGZTEZDP5EVGZOGUN6WRE5J3FI3ECWXVHZEB5HA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: <nil> Jun 25 13:01:51.053961520 (51h51m25.824602846s) [noise=true , address=45.91.116.12:5226 , piece_id=NSYZERWFIN5CMEX3BIV5NXLOAIFH3HWY5EQY2MLK3F2ES3LBKEVA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:5226): protocol: rpc: dial tcp 45.91.116.12:5226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054009372 (51h51m25.824650707s) [noise=true , address=77.90.31.17:4772 , piece_id=PKPQYYK3AP6BTTRW7NNB4NSCJASN6PDIFFZ5HBNM6YEEBFKZVUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054041034 (51h51m25.824682369s) [noise=true , address=77.90.31.21:4878 , piece_id=Z22HC22H4X3G6UQHQSLJOXRTQ54U4PXVBAHA5C22YHO7BFP7BUZA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054078988 (51h51m25.824720323s) [noise=true , address=77.90.31.18:4806 , piece_id=S4EKNUIRGPWMUCTZF4FQHA6RBYSBK5HRINHRF7VSJQ3ZOT7HZYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054109858 (51h51m25.824751193s) [noise=true , address=45.91.116.5:5298 , piece_id=2BSJEDM3GKA7GXK2HJSYY3PGO5YORAEKKOAKI2SXS3CRLEJJSQBA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: rpc: dial tcp 45.91.116.5:5298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054146298 (51h51m25.824787623s) [noise=true , address=77.90.31.14:4663 , piece_id=4Z7H3SH5HDKOLAXANUJEEDOM7ELB3I7X6N2M3UFVMJQKBQZZ7REQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: rpc: dial tcp 77.90.31.14:4663: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054174773 (51h51m25.824816108s) [noise=true , address=45.91.116.9:5261 , piece_id=N5AX5K6L6FBTUEB3ZBDPJK4MZSOK7PVHWANLV4FMP2P4YFFH5FBQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:5261): protocol: write tcp 188.166.225.170:48522->45.91.116.9:5261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054198679 (51h51m25.824840004s) [noise=true , address=45.91.116.11:4119 , piece_id=HCH7HK5DNKKCIDY777JHRIODOTD2KGGX2AK3KINQDSG4L5Y7RBRQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054214630 (51h51m25.824855965s) [noise=true , address=77.90.31.12:4592 , piece_id=755TNH7RCWX7LTNQAP5JFOPVFXSNPCUXM7ITVDLC5ITX2PE5WYPQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054242735 (51h51m25.82488407s) [noise=true , address=45.91.116.14:4190 , piece_id=WS6JVZWH5CTTXHLE5HDWLGFXKUYD5BXT3YKX6FXFY3YDGFFHTA2Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054252153 (51h51m25.824893488s) [noise=true , address=77.90.31.10:4520 , piece_id=VGPLDMWQH3BLAAGPFE5IU3HVAJ25C4DPMEY2TSAAJBFRZQOVFFFA, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054273424 (51h51m25.824914749s) [noise=true , address=77.90.31.16:4733 , piece_id=MZUAL7GUO2IWC4ZYXMHXXLTIX4NJELCQL6WFLPULNP3YGMKUZG6Q, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054287702 (51h51m25.824929037s) [noise=true , address=77.90.31.19:4841 , piece_id=C7WPSPITV6KAIQFAQMLDFJ6QX2K4ANTJ6WTYA2QBNUEC636OBH3A, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054292161 (51h51m25.824933486s) [noise=true , address=77.90.31.13:4626 , piece_id=LVPEQZCSQTLIZAKLZGDTR5JTCL2VEEXWNKG7LLEZVLEIUP2L4XUQ, seg_pos={0 0}, upload=41394]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: rpc: dial tcp 77.90.31.13:4626: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:51.054325435 (51h51m25.82496676s) [seg_pos={0 0}, upload=41394]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Jun 25 13:01:51.285654058 (51h51m26.056295383s) [upload=41394]: Done uploading segments... Jun 25 13:01:51.285679077 (51h51m26.056320402s) [upload=41394]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 30.234175116s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 13.132159016s FAIL (2025-06-25 13:01:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== 0 0}, upload=41382]: Done begin upload segment. Jun 25 13:00:32.553089489 (51h50m7.323730824s) [noise=true , address=45.91.116.10:4082 , piece_id=HMONZIGUQ3NM2PPOE4KQSTSHBLEUIVBBLSHADYXIT4RQQ6LIDZJQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.553094939 (51h50m7.323736274s) [noise=true , address=77.90.31.11:4555 , piece_id=P5ILWSOHIFCTEA6I2WC3GWMQN7ZT3K2BEG4AD7VONVMZX4HKNRWQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.553141158 (51h50m7.323782534s) [noise=true , address=77.90.31.16:4734 , piece_id=RNZAC4SQ7YQQSWF6OSCH2QDJNQIDABFKS54IO2FSJJRLSEPU4PZQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.553948663 (51h50m7.324590048s) [noise=true , address=104.36.82.12:4149 , piece_id=NQHBGHLUUGN5BDVOVXQYWR44ZIMUZNYTG3MYWFRWX2KZAD7L4DXA, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.553967599 (51h50m7.324608964s) [noise=true , address=104.36.82.6:5874 , piece_id=NTLYKXSCRR7WCNMFM3TW7WA7FVUVYIOR3KXHXIB6G4SZ3X7SAVFQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.554690178 (51h50m7.325331554s) [noise=true , address=104.36.82.13:4186 , piece_id=JKS5KMR3SST2OTA5KVVX2FDME7DZVFMAALWV4KB73MHJVUJ5LB2A, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.555251396 (51h50m7.325892731s) [noise=true , address=104.36.82.15:4258 , piece_id=ZBXEMZ2IEUDZTE6IBQAQMIPG6MBXUFLXUEF55L6P2WMGZS4ZEO3A, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.555805679 (51h50m7.326447025s) [noise=true , address=77.90.31.10:4518 , piece_id=373JETHZO7VGHYSEYP7JL76MGEBFXIS5V6W735JMHSSALM3OGQZQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.556322561 (51h50m7.326963906s) [noise=true , address=45.91.116.14:4189 , piece_id=7LKUDCAEKP4NX6WO52EZ2JPIYQFOS7UES2P5AQZ6YIQCOKGUBCIA, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.556793241 (51h50m7.327434577s) [noise=true , address=77.90.31.21:4879 , piece_id=LG2DN53ZN3NK3L4PVKRSP22LZVHFZNXARD6T57EFAEGDJREA5Z3Q, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.557201172 (51h50m7.327842517s) [noise=true , address=77.90.31.12:4589 , piece_id=5R7ZWFJV737IRPPZJP6TNZLLBMDBYELBP2V4IMMQQGBYU3G6VEGA, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.557614843 (51h50m7.328256188s) [noise=true , address=77.90.31.18:4806 , piece_id=WV7KJOJ53ANUP35CY6HF2SUJWCYHO2AG3TVIPIIS6QCX2MMB72EQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.558041499 (51h50m7.328682865s) [noise=true , address=77.90.31.13:4627 , piece_id=F2DF6ZHSZND4A65STV43UTCENMZNIFEMXESB2SUL3OZNRNJEZXNQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.558714022 (51h50m7.329355357s) [noise=true , address=45.91.116.6:4974 , piece_id=7BQXLYXOS6ISJK57OH6SLTBSDQIFB56QLTKMGS763GUBIWJXIJSQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.559140819 (51h50m7.329782164s) [noise=true , address=77.90.31.21:4878 , piece_id=SRYEUKLANF37RORZM5KSASL2DK3QOVOQ6C5RD57SBLO3FRU4OYXA, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.559524101 (51h50m7.330165436s) [noise=true , address=77.90.31.13:4628 , piece_id=OD5R6EF4ZJYSNXN7RVV4PGKWQK44Q3ZFWPWXFM4BQBRKTIK3YRFA, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:34.248122012 (51h50m9.018763347s) [upload=41382]: Next returned nil segment Jun 25 13:00:34.248140026 (51h50m9.018781361s) [upload=41382]: Waiting for error group managing segments... Jun 25 13:00:34.540357260 (51h50m9.310998595s) [noise=true , address=104.36.82.12:4149 , piece_id=NQHBGHLUUGN5BDVOVXQYWR44ZIMUZNYTG3MYWFRWX2KZAD7L4DXA, seg_pos={0 0}, upload=41382]: Done uploading piece. err: <nil> Jun 25 13:00:34.563871702 (51h50m9.334513037s) [noise=true , address=104.36.82.13:4186 , piece_id=JKS5KMR3SST2OTA5KVVX2FDME7DZVFMAALWV4KB73MHJVUJ5LB2A, seg_pos={0 0}, upload=41382]: Done uploading piece. err: <nil> Jun 25 13:00:34.601000791 (51h50m9.371642126s) [noise=true , address=104.36.82.15:4258 , piece_id=ZBXEMZ2IEUDZTE6IBQAQMIPG6MBXUFLXUEF55L6P2WMGZS4ZEO3A, seg_pos={0 0}, upload=41382]: Done uploading piece. err: <nil> Jun 25 13:00:34.625196943 (51h50m9.395838278s) [noise=true , address=45.91.116.6:4974 , piece_id=7BQXLYXOS6ISJK57OH6SLTBSDQIFB56QLTKMGS763GUBIWJXIJSQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: <nil> Jun 25 13:00:34.710040344 (51h50m9.480681679s) [noise=true , address=45.91.116.14:4189 , piece_id=7LKUDCAEKP4NX6WO52EZ2JPIYQFOS7UES2P5AQZ6YIQCOKGUBCIA, seg_pos={0 0}, upload=41382]: Done uploading piece. err: <nil> Jun 25 13:00:34.717077183 (51h50m9.487718518s) [noise=true , address=45.91.116.10:4082 , piece_id=HMONZIGUQ3NM2PPOE4KQSTSHBLEUIVBBLSHADYXIT4RQQ6LIDZJQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: <nil> Jun 25 13:00:34.958009580 (51h50m9.728650915s) [noise=true , address=104.36.82.6:5874 , piece_id=NTLYKXSCRR7WCNMFM3TW7WA7FVUVYIOR3KXHXIB6G4SZ3X7SAVFQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: <nil> Jun 25 13:01:01.646394582 (51h50m36.417035917s) [noise=true , address=77.90.31.16:4734 , piece_id=RNZAC4SQ7YQQSWF6OSCH2QDJNQIDABFKS54IO2FSJJRLSEPU4PZQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:45654->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:45654->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45654->77.90.31.16:4734: use of closed network connection Jun 25 13:01:01.646639196 (51h50m36.417280531s) [noise=true , address=77.90.31.11:4555 , piece_id=P5ILWSOHIFCTEA6I2WC3GWMQN7ZT3K2BEG4AD7VONVMZX4HKNRWQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:52694->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:52694->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52694->77.90.31.11:4555: use of closed network connection Jun 25 13:01:01.646645948 (51h50m36.417287283s) [noise=true , address=77.90.31.21:4878 , piece_id=SRYEUKLANF37RORZM5KSASL2DK3QOVOQ6C5RD57SBLO3FRU4OYXA, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: read tcp 188.166.225.170:37626->77.90.31.21:4878: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:01.646688190 (51h50m36.417329526s) [noise=true , address=77.90.31.18:4806 , piece_id=WV7KJOJ53ANUP35CY6HF2SUJWCYHO2AG3TVIPIIS6QCX2MMB72EQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: read tcp 188.166.225.170:38212->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:01.646723128 (51h50m36.417364463s) [noise=true , address=77.90.31.21:4879 , piece_id=LG2DN53ZN3NK3L4PVKRSP22LZVHFZNXARD6T57EFAEGDJREA5Z3Q, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 188.166.225.170:37676->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:01.646741554 (51h50m36.417382889s) [noise=true , address=77.90.31.12:4589 , piece_id=5R7ZWFJV737IRPPZJP6TNZLLBMDBYELBP2V4IMMQQGBYU3G6VEGA, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: read tcp 188.166.225.170:53064->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:01.646757965 (51h50m36.41739929s) [noise=true , address=77.90.31.13:4627 , piece_id=F2DF6ZHSZND4A65STV43UTCENMZNIFEMXESB2SUL3OZNRNJEZXNQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 188.166.225.170:40854->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:01.646782051 (51h50m36.417423377s) [noise=true , address=77.90.31.13:4628 , piece_id=OD5R6EF4ZJYSNXN7RVV4PGKWQK44Q3ZFWPWXFM4BQBRKTIK3YRFA, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:48956->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:01.646812220 (51h50m36.417453555s) [noise=true , address=77.90.31.10:4518 , piece_id=373JETHZO7VGHYSEYP7JL76MGEBFXIS5V6W735JMHSSALM3OGQZQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: read tcp 188.166.225.170:56344->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:01.646836498 (51h50m36.417477833s) [seg_pos={0 0}, upload=41382]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jun 25 13:01:01.883780495 (51h50m36.65442183s) [upload=41382]: Done uploading segments... Jun 25 13:01:01.883806494 (51h50m36.65444782s) [upload=41382]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 13.132176371s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.239087014s FAIL (2025-06-25 13:01:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== 0 0}, upload=41382]: Done begin upload segment. Jun 25 13:00:32.553089489 (51h50m7.323730824s) [noise=true , address=45.91.116.10:4082 , piece_id=HMONZIGUQ3NM2PPOE4KQSTSHBLEUIVBBLSHADYXIT4RQQ6LIDZJQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.553094939 (51h50m7.323736274s) [noise=true , address=77.90.31.11:4555 , piece_id=P5ILWSOHIFCTEA6I2WC3GWMQN7ZT3K2BEG4AD7VONVMZX4HKNRWQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.553141158 (51h50m7.323782534s) [noise=true , address=77.90.31.16:4734 , piece_id=RNZAC4SQ7YQQSWF6OSCH2QDJNQIDABFKS54IO2FSJJRLSEPU4PZQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.553948663 (51h50m7.324590048s) [noise=true , address=104.36.82.12:4149 , piece_id=NQHBGHLUUGN5BDVOVXQYWR44ZIMUZNYTG3MYWFRWX2KZAD7L4DXA, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.553967599 (51h50m7.324608964s) [noise=true , address=104.36.82.6:5874 , piece_id=NTLYKXSCRR7WCNMFM3TW7WA7FVUVYIOR3KXHXIB6G4SZ3X7SAVFQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.554690178 (51h50m7.325331554s) [noise=true , address=104.36.82.13:4186 , piece_id=JKS5KMR3SST2OTA5KVVX2FDME7DZVFMAALWV4KB73MHJVUJ5LB2A, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.555251396 (51h50m7.325892731s) [noise=true , address=104.36.82.15:4258 , piece_id=ZBXEMZ2IEUDZTE6IBQAQMIPG6MBXUFLXUEF55L6P2WMGZS4ZEO3A, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.555805679 (51h50m7.326447025s) [noise=true , address=77.90.31.10:4518 , piece_id=373JETHZO7VGHYSEYP7JL76MGEBFXIS5V6W735JMHSSALM3OGQZQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.556322561 (51h50m7.326963906s) [noise=true , address=45.91.116.14:4189 , piece_id=7LKUDCAEKP4NX6WO52EZ2JPIYQFOS7UES2P5AQZ6YIQCOKGUBCIA, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.556793241 (51h50m7.327434577s) [noise=true , address=77.90.31.21:4879 , piece_id=LG2DN53ZN3NK3L4PVKRSP22LZVHFZNXARD6T57EFAEGDJREA5Z3Q, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.557201172 (51h50m7.327842517s) [noise=true , address=77.90.31.12:4589 , piece_id=5R7ZWFJV737IRPPZJP6TNZLLBMDBYELBP2V4IMMQQGBYU3G6VEGA, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.557614843 (51h50m7.328256188s) [noise=true , address=77.90.31.18:4806 , piece_id=WV7KJOJ53ANUP35CY6HF2SUJWCYHO2AG3TVIPIIS6QCX2MMB72EQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.558041499 (51h50m7.328682865s) [noise=true , address=77.90.31.13:4627 , piece_id=F2DF6ZHSZND4A65STV43UTCENMZNIFEMXESB2SUL3OZNRNJEZXNQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.558714022 (51h50m7.329355357s) [noise=true , address=45.91.116.6:4974 , piece_id=7BQXLYXOS6ISJK57OH6SLTBSDQIFB56QLTKMGS763GUBIWJXIJSQ, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.559140819 (51h50m7.329782164s) [noise=true , address=77.90.31.21:4878 , piece_id=SRYEUKLANF37RORZM5KSASL2DK3QOVOQ6C5RD57SBLO3FRU4OYXA, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:32.559524101 (51h50m7.330165436s) [noise=true , address=77.90.31.13:4628 , piece_id=OD5R6EF4ZJYSNXN7RVV4PGKWQK44Q3ZFWPWXFM4BQBRKTIK3YRFA, seg_pos={0 0}, upload=41382]: Uploading piece... Jun 25 13:00:34.248122012 (51h50m9.018763347s) [upload=41382]: Next returned nil segment Jun 25 13:00:34.248140026 (51h50m9.018781361s) [upload=41382]: Waiting for error group managing segments... Jun 25 13:00:34.540357260 (51h50m9.310998595s) [noise=true , address=104.36.82.12:4149 , piece_id=NQHBGHLUUGN5BDVOVXQYWR44ZIMUZNYTG3MYWFRWX2KZAD7L4DXA, seg_pos={0 0}, upload=41382]: Done uploading piece. err: <nil> Jun 25 13:00:34.563871702 (51h50m9.334513037s) [noise=true , address=104.36.82.13:4186 , piece_id=JKS5KMR3SST2OTA5KVVX2FDME7DZVFMAALWV4KB73MHJVUJ5LB2A, seg_pos={0 0}, upload=41382]: Done uploading piece. err: <nil> Jun 25 13:00:34.601000791 (51h50m9.371642126s) [noise=true , address=104.36.82.15:4258 , piece_id=ZBXEMZ2IEUDZTE6IBQAQMIPG6MBXUFLXUEF55L6P2WMGZS4ZEO3A, seg_pos={0 0}, upload=41382]: Done uploading piece. err: <nil> Jun 25 13:00:34.625196943 (51h50m9.395838278s) [noise=true , address=45.91.116.6:4974 , piece_id=7BQXLYXOS6ISJK57OH6SLTBSDQIFB56QLTKMGS763GUBIWJXIJSQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: <nil> Jun 25 13:00:34.710040344 (51h50m9.480681679s) [noise=true , address=45.91.116.14:4189 , piece_id=7LKUDCAEKP4NX6WO52EZ2JPIYQFOS7UES2P5AQZ6YIQCOKGUBCIA, seg_pos={0 0}, upload=41382]: Done uploading piece. err: <nil> Jun 25 13:00:34.717077183 (51h50m9.487718518s) [noise=true , address=45.91.116.10:4082 , piece_id=HMONZIGUQ3NM2PPOE4KQSTSHBLEUIVBBLSHADYXIT4RQQ6LIDZJQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: <nil> Jun 25 13:00:34.958009580 (51h50m9.728650915s) [noise=true , address=104.36.82.6:5874 , piece_id=NTLYKXSCRR7WCNMFM3TW7WA7FVUVYIOR3KXHXIB6G4SZ3X7SAVFQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: <nil> Jun 25 13:01:01.646394582 (51h50m36.417035917s) [noise=true , address=77.90.31.16:4734 , piece_id=RNZAC4SQ7YQQSWF6OSCH2QDJNQIDABFKS54IO2FSJJRLSEPU4PZQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:45654->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:45654->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45654->77.90.31.16:4734: use of closed network connection Jun 25 13:01:01.646639196 (51h50m36.417280531s) [noise=true , address=77.90.31.11:4555 , piece_id=P5ILWSOHIFCTEA6I2WC3GWMQN7ZT3K2BEG4AD7VONVMZX4HKNRWQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:52694->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:52694->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52694->77.90.31.11:4555: use of closed network connection Jun 25 13:01:01.646645948 (51h50m36.417287283s) [noise=true , address=77.90.31.21:4878 , piece_id=SRYEUKLANF37RORZM5KSASL2DK3QOVOQ6C5RD57SBLO3FRU4OYXA, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: read tcp 188.166.225.170:37626->77.90.31.21:4878: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:01.646688190 (51h50m36.417329526s) [noise=true , address=77.90.31.18:4806 , piece_id=WV7KJOJ53ANUP35CY6HF2SUJWCYHO2AG3TVIPIIS6QCX2MMB72EQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: read tcp 188.166.225.170:38212->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:01.646723128 (51h50m36.417364463s) [noise=true , address=77.90.31.21:4879 , piece_id=LG2DN53ZN3NK3L4PVKRSP22LZVHFZNXARD6T57EFAEGDJREA5Z3Q, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 188.166.225.170:37676->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:01.646741554 (51h50m36.417382889s) [noise=true , address=77.90.31.12:4589 , piece_id=5R7ZWFJV737IRPPZJP6TNZLLBMDBYELBP2V4IMMQQGBYU3G6VEGA, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: read tcp 188.166.225.170:53064->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:01.646757965 (51h50m36.41739929s) [noise=true , address=77.90.31.13:4627 , piece_id=F2DF6ZHSZND4A65STV43UTCENMZNIFEMXESB2SUL3OZNRNJEZXNQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 188.166.225.170:40854->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:01.646782051 (51h50m36.417423377s) [noise=true , address=77.90.31.13:4628 , piece_id=OD5R6EF4ZJYSNXN7RVV4PGKWQK44Q3ZFWPWXFM4BQBRKTIK3YRFA, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:48956->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:01.646812220 (51h50m36.417453555s) [noise=true , address=77.90.31.10:4518 , piece_id=373JETHZO7VGHYSEYP7JL76MGEBFXIS5V6W735JMHSSALM3OGQZQ, seg_pos={0 0}, upload=41382]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: read tcp 188.166.225.170:56344->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 25 13:01:01.646836498 (51h50m36.417477833s) [seg_pos={0 0}, upload=41382]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jun 25 13:01:01.883780495 (51h50m36.65442183s) [upload=41382]: Done uploading segments... Jun 25 13:01:01.883806494 (51h50m36.65444782s) [upload=41382]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.239115989s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.928340151s FAIL (2025-06-25 08:44:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.928359699s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)