Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X..........................................X] duration: 30.263518619s trace: https://jaeger.dev.storj.io/trace/12a968c1abea4bf9 FAIL (2025-11-11 16:09:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== TQJ4P5T7C2BPFRDSTP4JHJIUITAIK5LV3RZRQ, seg_pos={0 0}, upload=49418]: Uploading piece... Nov 11 16:08:59.582327522 (56h4m58.133797983s) [noise=true , address=145.79.227.11:4730 , piece_id=MDBZFCBE35WNTVFMJBISHKJDJOKD22KFRSXMWGC7ZVGNYJMUCXMA, seg_pos={0 0}, upload=49418]: Uploading piece... Nov 11 16:08:59.582440517 (56h4m58.133910978s) [noise=true , address=77.90.31.11:4555 , piece_id=C6M7M2QN6V7KBXVXATFFWKRQOADS2JWJXTJ2TA65POVXDKKY4Y6Q, seg_pos={0 0}, upload=49418]: Uploading piece... Nov 11 16:08:59.582767118 (56h4m58.134237579s) [noise=true , address=145.79.227.20:4439 , piece_id=2AZ5LDWBWBOA6QDZ563IJD5K4B676RMLI5MO2XAIO3XZMP6BLSAA, seg_pos={0 0}, upload=49418]: Uploading piece... Nov 11 16:08:59.582934197 (56h4m58.134404658s) [noise=true , address=145.79.227.23:4546 , piece_id=3GJAFODINGGSXH65E7Q4M5P5FDH5SNVE52CT6OQH6PN3NPQJDSEQ, seg_pos={0 0}, upload=49418]: Uploading piece... Nov 11 16:08:59.583139287 (56h4m58.134609748s) [noise=true , address=145.79.227.14:4837 , piece_id=ZPC6XXXZN7KDWNARROKROYFQL6JMSCU3IWPFKM6XNH63M4RPEILA, seg_pos={0 0}, upload=49418]: Uploading piece... Nov 11 16:09:01.275898795 (56h4m59.827369256s) [upload=49418]: Next returned nil segment Nov 11 16:09:01.275920286 (56h4m59.827390747s) [upload=49418]: Waiting for error group managing segments... Nov 11 16:09:01.699831625 (56h5m0.251302086s) [noise=true , address=45.91.116.9:4261 , piece_id=N23BCGQUIKMH76XTQJ4P5T7C2BPFRDSTP4JHJIUITAIK5LV3RZRQ, seg_pos={0 0}, upload=49418]: Done uploading piece. err: <nil> Nov 11 16:09:01.765339256 (56h5m0.316809717s) [noise=true , address=77.90.31.11:4555 , piece_id=C6M7M2QN6V7KBXVXATFFWKRQOADS2JWJXTJ2TA65POVXDKKY4Y6Q, seg_pos={0 0}, upload=49418]: Done uploading piece. err: <nil> Nov 11 16:09:01.786655106 (56h5m0.338125567s) [noise=true , address=77.90.31.12:4590 , piece_id=QGYTQ4I46NO6ZNEG7DHBYBLUOROIAQPXPJDXPMLLGSJQFGS7W4UQ, seg_pos={0 0}, upload=49418]: Done uploading piece. err: <nil> Nov 11 16:09:27.054329351 (56h5m25.605799832s) [noise=true , address=145.79.227.11:4730 , piece_id=MDBZFCBE35WNTVFMJBISHKJDJOKD22KFRSXMWGC7ZVGNYJMUCXMA, seg_pos={0 0}, upload=49418]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:54962->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:54962->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54962->145.79.227.11:4730: use of closed network connection Nov 11 16:09:27.054410526 (56h5m25.605880987s) [noise=true , address=145.79.227.24:4582 , piece_id=S7FWB75RPRZPTPRC7O5GNOLI3PXJQA3AS6XGTTM75FOLXHST5BDA, seg_pos={0 0}, upload=49418]: Done uploading piece. err: ecclient: upload failed (node:1YLhczXWKCwzCQxM97Q77HpiWNMe5E15Nv679FZo5uHw6MRXa, address:145.79.227.24:4582): protocol: write tcp 188.166.225.170:45206->145.79.227.24:4582: use of closed network connection; write tcp 188.166.225.170:45206->145.79.227.24:4582: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45206->145.79.227.24:4582: use of closed network connection Nov 11 16:09:27.054503613 (56h5m25.605974074s) [noise=true , address=145.79.227.6:4258 , piece_id=EERE6AA237LNMAHODY2I4UC66VVS6KK4ZGJUHQVF4FT2AW36CYQA, seg_pos={0 0}, upload=49418]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 188.166.225.170:32826->145.79.227.6:4258: use of closed network connection; write tcp 188.166.225.170:32826->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32826->145.79.227.6:4258: use of closed network connection Nov 11 16:09:27.054611839 (56h5m25.6060823s) [noise=true , address=145.79.227.9:4366 , piece_id=AFBLCNRMJHNJVTJBZ7JLBQINB2I7TTS26JM3NNSGUWTJAXU46I5Q, seg_pos={0 0}, upload=49418]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37908->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37908->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37908->145.79.227.9:4366: use of closed network connection Nov 11 16:09:27.054817130 (56h5m25.606287601s) [noise=true , address=145.79.227.10:4402 , piece_id=UZIO6CHMP64MMB754NY4YTMFGGGJF57CAA56QRUT6BA2PANLG4JA, seg_pos={0 0}, upload=49418]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:49942->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:49942->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49942->145.79.227.10:4402: use of closed network connection Nov 11 16:09:27.054903936 (56h5m25.606374407s) [noise=true , address=145.79.227.23:4546 , piece_id=3GJAFODINGGSXH65E7Q4M5P5FDH5SNVE52CT6OQH6PN3NPQJDSEQ, seg_pos={0 0}, upload=49418]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:50972->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:50972->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50972->145.79.227.23:4546: use of closed network connection Nov 11 16:09:27.055031057 (56h5m25.606501518s) [noise=true , address=145.79.227.14:4837 , piece_id=ZPC6XXXZN7KDWNARROKROYFQL6JMSCU3IWPFKM6XNH63M4RPEILA, seg_pos={0 0}, upload=49418]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:56288->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:56288->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56288->145.79.227.14:4837: use of closed network connection Nov 11 16:09:27.055153871 (56h5m25.606624332s) [noise=true , address=145.79.227.20:4439 , piece_id=2AZ5LDWBWBOA6QDZ563IJD5K4B676RMLI5MO2XAIO3XZMP6BLSAA, seg_pos={0 0}, upload=49418]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: write tcp 188.166.225.170:58700->145.79.227.20:4439: use of closed network connection; write tcp 188.166.225.170:58700->145.79.227.20:4439: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58700->145.79.227.20:4439: use of closed network connection Nov 11 16:09:27.055319387 (56h5m25.606789838s) [noise=true , address=145.79.227.22:4510 , piece_id=B4NKOC6JYG7MO6GIPLJSLC3YTE4NPWHGWLOS4MKHEUILDZDDK4TQ, seg_pos={0 0}, upload=49418]: Done uploading piece. err: ecclient: upload failed (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9, address:145.79.227.22:4510): protocol: write tcp 188.166.225.170:46704->145.79.227.22:4510: use of closed network connection; write tcp 188.166.225.170:46704->145.79.227.22:4510: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46704->145.79.227.22:4510: use of closed network connection Nov 11 16:09:27.055409398 (56h5m25.606879859s) [noise=true , address=145.79.227.15:4874 , piece_id=OPHMUZXPLUJ4AZJQWRR35YP6BUTRXO6X22F26BOT4KHAXUTPXADQ, seg_pos={0 0}, upload=49418]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:51200->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:09:27.055460305 (56h5m25.606930776s) [noise=true , address=145.79.227.7:4295 , piece_id=TMHGBHP4S53KZCYQOR257IF3U4CS5GD44T56C5VXB77MRZ4OM2KA, seg_pos={0 0}, upload=49418]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:43104->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:43104->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43104->145.79.227.7:4295: use of closed network connection Nov 11 16:09:27.055793689 (56h5m25.60726415s) [noise=true , address=145.79.227.8:4329 , piece_id=HQG2SUOZSSZAWT4P43EFMB6A6XEE25KLWJVJVERP5OHNXQSCZGKQ, seg_pos={0 0}, upload=49418]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:46454->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:09:27.056044989 (56h5m25.60751546s) [noise=true , address=145.79.227.26:4653 , piece_id=J337KHUBUUE26DTE2XOIQDSQUB7LTPECERHODVGH7OY3XE65SKNA, seg_pos={0 0}, upload=49418]: Done uploading piece. err: ecclient: upload failed (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ, address:145.79.227.26:4653): protocol: read tcp 188.166.225.170:34444->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:09:27.056089854 (56h5m25.607560315s) [seg_pos={0 0}, upload=49418]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Nov 11 16:09:27.314812176 (56h5m25.866282637s) [upload=49418]: Done uploading segments... Nov 11 16:09:27.314851021 (56h5m25.866321482s) [upload=49418]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 30.263531924s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 12.228771965s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 16:08:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 12.228798226s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 9.326199393s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 16:08:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 9.326232386s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..................X........................................] duration: 9.716327212s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 16:07:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 9.71635311s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X..................X.......................................] duration: 9.244924189s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 16:06:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 9.24494074s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X..................X......................................] duration: 9.874079721s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 16:06:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 9.874107834s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X..................X.....................................] duration: 9.346877808s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 16:05:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 9.346909377s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X..................X....................................] duration: 10.473019125s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 16:04:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 10.473062176s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X..................X...................................] duration: 8.431112435s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 16:04:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.431222855s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X..................X..................................] duration: 11.186897674s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 16:03:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 11.18693231s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X..................X.................................] duration: 10.398449619s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 16:02:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 10.398494625s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X.......X..................X................................] duration: 10.887555025s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 16:00:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 10.887587608s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X.......X..................X...............................] duration: 12.11449s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 16:00:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 12.114518784s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X.......X..................X..............................] duration: 8.949116795s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:59:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.949150138s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X.......X..................X.............................] duration: 8.111408464s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:58:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.111434656s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X.......X..................X............................] duration: 9.061638127s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:57:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 9.061664207s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X.......X..................X...........................] duration: 8.099520098s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:56:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.099551939s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X.......X..................X..........................] duration: 8.23383851s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:56:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.233872225s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X.......X..................X.........................] duration: 8.69599272s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:55:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.696025543s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X.......X..................X........................] duration: 9.274431956s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:54:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 9.274463224s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X.......X..................X.......................] duration: 8.901335633s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:53:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.901374456s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X.......X..................X......................] duration: 8.077616566s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:53:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.077701018s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X.......X..................X.....................] duration: 9.588370135s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:51:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 9.588399021s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X.......X..................X....................] duration: 8.896407298s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:51:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.89644027s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X.......X..................X...................] duration: 7.879968098s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:50:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.88000112s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X.......X..................X..................] duration: 7.825614187s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:50:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.825646017s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X.......X..................X.................] duration: 7.208347039s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:49:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.208381043s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X.......X..................X................] duration: 9.172621217s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:48:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 9.172648089s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X.......X..................X...............] duration: 7.760767995s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:47:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.760800087s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X.......X..................X..............] duration: 8.910515954s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:47:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.910546043s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X.......X..................X.............] duration: 8.888887362s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:46:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.888921317s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X.......X..................X............] duration: 7.520532727s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:45:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.520563947s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X.......X..................X...........] duration: 7.983729603s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:44:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.983761754s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X.......X..................X..........] duration: 8.089235331s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:43:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.089263915s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X.......X..................X.........] duration: 8.21490908s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:43:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 8.214931252s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................X.......X..................X........] duration: 6.514013535s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:41:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.514045153s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................X.......X..................X.......] duration: 6.816180836s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:40:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.816210523s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................X.......X..................X......] duration: 7.112085008s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:40:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.112115056s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................X.......X..................X.....] duration: 7.537042722s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:39:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.537074122s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................X.......X..................X....] duration: 7.324953974s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:39:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.324990924s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................X.......X..................X...] duration: 6.609847865s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:38:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.609878974s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................X.......X..................X..] duration: 11.190905511s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:37:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 11.190943745s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................X.......X..................X.] duration: 10.899057936s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:36:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 10.899090437s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................X.......X..................X] duration: 30.254808694s trace: https://jaeger.dev.storj.io/trace/10157ce761c74c6b FAIL (2025-11-11 15:36:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== FKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.546928610 (55h32m10.098399102s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.547058117 (55h32m10.098528578s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.548041309 (55h32m10.09951176s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550410612 (55h32m10.101881063s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.550529628 (55h32m10.102000089s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551267172 (55h32m10.102737864s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.551649340 (55h32m10.103119801s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:11.552259933 (55h32m10.103730394s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Uploading piece... Nov 11 15:36:13.298219742 (55h32m11.849690203s) [upload=48942]: Next returned nil segment Nov 11 15:36:13.298244789 (55h32m11.84971524s) [upload=48942]: Waiting for error group managing segments... Nov 11 15:36:13.710008988 (55h32m12.261479449s) [noise=true , address=45.91.116.12:4227 , piece_id=IGKHCPPTTJVUMUHZPO5A7DEZKFT26UMKGBTTE4WYOPQOSKOMSB4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.719515751 (55h32m12.270986212s) [noise=true , address=77.90.31.18:4807 , piece_id=6PNWDKD2HABQI7746ESWUJZYHLQKKKNUL6XU2QPOLRKOOL4UIYEA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.744611669 (55h32m12.29608213s) [noise=true , address=77.90.31.19:4841 , piece_id=32T5QT3CK25IA24UTUJGUSX5WNSIQ5TVF7NPLMPIZ5KIPNNL2KJA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.753430622 (55h32m12.304901073s) [noise=true , address=45.91.116.13:4153 , piece_id=IRN5GLAGN7KTUNHFOTDQVGTUUW6TTCKV3HLOFKRQ3PGVLV4UEMGQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:13.802903650 (55h32m12.354374111s) [noise=true , address=45.91.116.4:4939 , piece_id=WVNUFFJFMYAD7CGWBK2D4DTJUBYOFAJMGQ7MFS5GPBGFWM3XO6YQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: <nil> Nov 11 15:36:40.535858606 (55h32m39.087329057s) [noise=true , address=145.79.227.24:4583 , piece_id=HXJOYWMSRS6C55INKQT2VOX5AGVGSLB2BWSDVMCEZRS46AAEJP2A, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60594->145.79.227.24:4583: use of closed network connection Nov 11 15:36:40.535895948 (55h32m39.087366419s) [noise=true , address=145.79.227.11:4732 , piece_id=BI5TOMIRFBKMW7SUEY5SWQBRCJJZECYQDDIUTKBMKY62AMJFEJYQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41758->145.79.227.11:4732: use of closed network connection Nov 11 15:36:40.536017860 (55h32m39.087488311s) [noise=true , address=145.79.227.11:4731 , piece_id=NCIP3BWKSBOQSOLD5K47PXI3Y4RCWBQFME5YCSZ2IOF25Z2J6JRQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47174->145.79.227.11:4731: use of closed network connection Nov 11 15:36:40.536089867 (55h32m39.087560328s) [noise=true , address=145.79.227.15:4873 , piece_id=AS3MOLHQRFRTBP4OYOMVH3O3R66JYA2X5V4AHB3WRW6GMKE2EPYA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:36:40.536159499 (55h32m39.08762996s) [noise=true , address=145.79.227.32:4835 , piece_id=LYKUMU4JPZXGGC22OD6TUXUYQL2E7L7CNIYJ4ZMDUOXS5V5CCY2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57696->145.79.227.32:4835: use of closed network connection Nov 11 15:36:40.536216368 (55h32m39.087686829s) [noise=true , address=145.79.227.4:4186 , piece_id=JCIEOGXNQSV5NNM2L4OPAIJK2X3TNLJSWV32BBO5NILSCUQ2DI4Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57060->145.79.227.4:4186: use of closed network connection Nov 11 15:36:40.536297963 (55h32m39.087768414s) [noise=true , address=145.79.227.17:4971 , piece_id=A4IY3PCFI6FM6EIHNQHHT4HCS3UMUDXNV7J4KS677RDBK7SALNSQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33824->145.79.227.17:4971: use of closed network connection Nov 11 15:36:40.536328311 (55h32m39.087798772s) [noise=true , address=145.79.227.25:4619 , piece_id=K4E56WCMZFWPE4DVTE3T7472EE4VHYQO4KCMHU6QKESLGOYTY7SA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1zuLJznAUFHkanuomswSd7JN6CkqPXso8Ak8e5E57eaEqvjodU, address:145.79.227.25:4619): protocol: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45240->145.79.227.25:4619: use of closed network connection Nov 11 15:36:40.536393525 (55h32m39.087863986s) [noise=true , address=145.79.227.3:4150 , piece_id=7ORA33NE7TQ737LQ5VYRUEFLRN7WT3MBHMXVRWQ6VHV2D3IDOKGA, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51512->145.79.227.3:4150: use of closed network connection Nov 11 15:36:40.536527009 (55h32m39.08799746s) [noise=true , address=145.79.227.26:4654 , piece_id=TLGEMEBG4LZAMAIO5S6RT6MQHRWBALUGJ4QQCZOZFRYW36FQGA2Q, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1KhBWMerZe9tUmzdVEYkBb6NmJ25dGP26Ea7CLjNJT5gdAeiah, address:145.79.227.26:4654): protocol: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36582->145.79.227.26:4654: use of closed network connection Nov 11 15:36:40.536593196 (55h32m39.088063657s) [noise=true , address=145.79.227.10:4401 , piece_id=5G5XRTXA5L4HHPTBLIAIKGEJLKTNHW3S6JKWLAM6KJW2KYO5NEKQ, seg_pos={0 0}, upload=48942]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37802->145.79.227.10:4401: use of closed network connection Nov 11 15:36:40.536629935 (55h32m39.088100397s) [seg_pos={0 0}, upload=48942]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 11 15:36:40.789363716 (55h32m39.340834187s) [upload=48942]: Done uploading segments... Nov 11 15:36:40.789407920 (55h32m39.340878381s) [upload=48942]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.254839974s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.......X..................] duration: 7.971743618s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:35:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.971781972s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X.......X.................] duration: 8.472845848s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:33:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 8.472880053s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................X.......X................] duration: 8.00810106s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:32:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 8.008132068s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.......X...............] duration: 7.805285253s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:32:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.805320711s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.......X..............] duration: 7.655990228s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:31:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.656014835s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X.......X.............] duration: 7.561038684s trace: https://jaeger.dev.storj.io/trace/699fb7b594fab73f FAIL (2025-11-11 15:30:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.553112139 (55h15m25.10458259s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.554777952 (55h15m25.106248413s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.558642440 (55h15m25.110112921s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.561870437 (55h15m25.113340918s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.564089733 (55h15m25.115560194s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.568218125 (55h15m25.119688596s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:26.570422101 (55h15m25.121892552s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Uploading piece... Nov 11 15:19:28.303126355 (55h15m26.854596816s) [upload=48693]: Next returned nil segment Nov 11 15:19:28.303147034 (55h15m26.854617495s) [upload=48693]: Waiting for error group managing segments... Nov 11 15:19:28.704963331 (55h15m27.256433792s) [noise=true , address=45.91.116.7:4010 , piece_id=PTUAZZCLAUSXOJJZQ24DWY7VDC74LVKSBLFHWWU6HENXFNCN7RHQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:28.747017731 (55h15m27.298488182s) [noise=true , address=45.91.116.6:4974 , piece_id=AT5SUAEQCXFJA37I6KDQK6BHFVVBWSJX45KF2HUXPMJHPEYLBMNQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:50.948406052 (55h15m49.499876513s) [noise=true , address=145.79.227.25:4617 , piece_id=TPJLP67MERBR4PEYS4FXLFOG5IIBJIOJAAUCLJLEWFXOTUSQWSAQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:52.001018970 (55h15m50.552489451s) [noise=true , address=145.79.227.26:4655 , piece_id=RYX3GBXC3WBYAD7TBIORST36NA247SNI7TQHOCHMXQACCOT5QNDA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: <nil> Nov 11 15:19:55.589637559 (55h15m54.141108021s) [noise=true , address=145.79.227.27:4691 , piece_id=V6QPMR5RO3OMXLYK6RHVTY75YKSX4NOYPLW5EBHQ25Q5NKUADWMA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg, address:145.79.227.27:4691): protocol: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48808->145.79.227.27:4691: use of closed network connection Nov 11 15:19:55.589768259 (55h15m54.14123872s) [noise=true , address=145.79.227.30:4763 , piece_id=EXGZKAHIJCZ6EZ32UOYSMXJZ4MKBHMOQV4JVOFSKLNJC5SJRODJQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37378->145.79.227.30:4763: use of closed network connection Nov 11 15:19:55.589989349 (55h15m54.14145981s) [noise=true , address=145.79.227.21:4474 , piece_id=W443NU2DFYHAMYNWE2FN3ZMCK2JKVCCCUJZQLSSGA5XMJLLJV3SA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590087106 (55h15m54.141557557s) [noise=true , address=145.79.227.15:4874 , piece_id=Z2SKEPM4IO3LIPH6UFC6V5VWZZSEIWLZJIKS3ILAQNQK5MBOS5PA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47456->145.79.227.15:4874: use of closed network connection Nov 11 15:19:55.590202295 (55h15m54.141672756s) [noise=true , address=145.79.227.16:4911 , piece_id=OTX3AEM3CFG3STMAXRDIKMPRDPGHA2IDSZSYIL43F7YA3Q35NHLA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43142->145.79.227.16:4911: use of closed network connection Nov 11 15:19:55.590254004 (55h15m54.141724465s) [noise=true , address=145.79.227.11:4732 , piece_id=EU4IZJ54JRELRLHNHIKOPEYHF7O7WF7WV7OW2O6BKCXUBRY3W6YQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58300->145.79.227.11:4732: use of closed network connection Nov 11 15:19:55.590308667 (55h15m54.141779118s) [noise=true , address=145.79.227.10:4403 , piece_id=ETWCSR7LBGC32TBKA6MKANFFT76IN3BRBKZS2XCKBTWS2TUSYE5A, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41802->145.79.227.10:4403: use of closed network connection Nov 11 15:19:55.590339796 (55h15m54.141810258s) [noise=true , address=145.79.227.23:4545 , piece_id=APEKWTWPUPPD7Q6OGA5RO65PFPLFEJOIHPNJAA6JTWPSTQ7WYUSQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47162->145.79.227.23:4545: use of closed network connection Nov 11 15:19:55.590454555 (55h15m54.141925016s) [noise=true , address=145.79.227.13:4803 , piece_id=SMQMCU7HFF47ONE64FDQKFQE3JZINSOIRXOBZHTIU326WCZDZBVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43794->145.79.227.13:4803: use of closed network connection Nov 11 15:19:55.590552321 (55h15m54.142022782s) [noise=true , address=145.79.227.4:4186 , piece_id=WOJVPITBL4FQKOUUSSVPGRVZ55GA2XECF54JYMWJWI7CDZMMZDVQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47372->145.79.227.4:4186: use of closed network connection Nov 11 15:19:55.590658984 (55h15m54.142129445s) [noise=true , address=145.79.227.3:4150 , piece_id=YJBTJJOH764H45IW477JAFAZBJJBHARHJMMHXCFDSCFUQITEL3WA, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Nov 11 15:19:55.590767170 (55h15m54.142237631s) [noise=true , address=145.79.227.27:4690 , piece_id=IXAXQ2C5H7XKGL5QKION3WHICWDK6D7PMFPJAXOPOAQ5K5PIQIYQ, seg_pos={0 0}, upload=48693]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52492->145.79.227.27:4690: use of closed network connection Nov 11 15:19:55.590804521 (55h15m54.142274982s) [seg_pos={0 0}, upload=48693]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Nov 11 15:19:55.838863650 (55h15m54.390334112s) [upload=48693]: Done uploading segments... Nov 11 15:19:55.838913065 (55h15m54.390383516s) [upload=48693]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 7.561069572s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)