Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [.....................................XXXX...................] duration: 6.106832564s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:53:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 6.106856449s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [......................................XXXX..................] duration: 4.839010431s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:52:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 4.839032813s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [.......................................XXXX.................] duration: 5.389568896s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:51:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.389599504s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [........................................XXXX................] duration: 5.942408562s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:51:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.942441065s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [.........................................XXXX...............] duration: 6.0099076s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:50:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 6.009951884s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [..........................................XXXX..............] duration: 5.44944267s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:49:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.449467927s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [...........................................XXXX.............] duration: 5.922824279s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:49:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.922871449s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [............................................XXXX............] duration: 4.834250582s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:48:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 4.834283534s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [.............................................XXXX...........] duration: 6.003028773s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:47:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 6.003061715s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [..............................................XXXX..........] duration: 4.882483734s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:46:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 4.882530874s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [...............................................XXXX.........] duration: 5.493108904s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:46:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.493129834s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [................................................XXXX........] duration: 6.078971381s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:45:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 6.079006449s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [.................................................XXXX.......] duration: 5.505643204s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:44:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.505674252s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [..................................................XXXX......] duration: 4.872362922s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:43:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 4.872396456s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [...................................................XXXX.....] duration: 5.446609785s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:42:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.44663921s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [X...................................................XXXX....] duration: 4.787094779s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:41:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 4.787123794s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [.X...................................................XXXX...] duration: 4.866540168s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:41:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 4.866569143s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [..X...................................................XXXX..] duration: 4.662655305s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:40:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 4.662683347s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [...X...................................................XXXX.] duration: 5.483985824s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:39:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.484002204s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 6 out of 60 is failed (threshold=3) [X...X...................................................XXXX] duration: 1m0.174173255s trace: https://jaeger.dev.storj.io/trace/66046224e1191e19 FAIL (2025-11-04 15:39:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== YY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.052795943 (123h58m8.748644686s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053377350 (123h58m8.749226103s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.053944791 (123h58m8.749793544s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054450924 (123h58m8.750299677s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.054715628 (123h58m8.750564381s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.055526011 (123h58m8.751374764s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056365930 (123h58m8.752214683s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.056899186 (123h58m8.752747939s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.057282727 (123h58m8.75313152s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Uploading piece... Nov 4 15:38:37.917468353 (123h58m9.613317106s) [upload=109193]: Next returned nil segment Nov 4 15:38:37.918380680 (123h58m9.614229433s) [upload=109193]: Waiting for error group managing segments... Nov 4 15:38:38.322335131 (123h58m10.018183884s) [noise=true , address=89.149.201.147:4017 , piece_id=A4JA4OIK4INGDTFYD4EMYL4YNVFJPLR2EBCSDUVT364W6DHM2EIA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.330089968 (123h58m10.025938721s) [noise=true , address=89.149.201.147:4006 , piece_id=AZCVSD3ZKTIQJC7J3AT3L65NCPBFJ5AJMR4P6OZGVVOXQJGHQ75A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.374527154 (123h58m10.070375907s) [noise=true , address=89.149.201.147:4019 , piece_id=GEFGB3Z7KGBMENBA27UK7FKRTNOXNXK5MDUI4ZGGQX53RZY7BNZA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.375198312 (123h58m10.071047065s) [noise=true , address=89.149.201.147:4024 , piece_id=3NPSZ76SDIYKH57ZDAFKKC2OS2HZT3OGKNFNUIYTS4NPV77XEO6A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.377488182 (123h58m10.073336926s) [noise=true , address=89.149.201.147:4009 , piece_id=QM2DBY6OM4KQUIRO5HT2COKF75RNQCEZXQ7F7XFRESPAQQ3WOCYQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.843998882 (123h58m10.539847625s) [noise=true , address=89.149.201.147:4010 , piece_id=XP5ORXPCRJVZZR5EA47VDFC4L2GFCZ2ELHIWFZFWXV57NSVQTU4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.854624365 (123h58m10.550473118s) [noise=true , address=89.149.201.147:4004 , piece_id=IXVFANRXUUSRC5HBNU5NZY3RGNZN7V6FVDNWEU55OQLRGE2MMXJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.867700775 (123h58m10.563549528s) [noise=true , address=185.167.204.205:4027 , piece_id=5ODQMMDN7CKKHKPAOCTU42CRZTZWWLDDUNYMZR3U6TLQPFYXUN5Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.880776112 (123h58m10.576624865s) [noise=true , address=185.167.204.205:4034 , piece_id=NQFAVWDCGRB7XNAOK2ASBKZL6KHTBYY4NAAUXUWKPJPBDDOP3CJA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.890514826 (123h58m10.586363589s) [noise=true , address=89.149.201.147:4012 , piece_id=Y23ICZNCUSGJG5Z5WUYNLGJ6Z2OMZPXYQKS3TJA3F7WM2D7ARMWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.893240436 (123h58m10.589089179s) [noise=true , address=185.167.204.206:4057 , piece_id=EYNXMCG6T4CKUAZR4WFVHJM7FWEN5GEGNGPS6SQ24KMS7AX47H7Q, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.896928368 (123h58m10.592777122s) [noise=true , address=89.149.201.147:4015 , piece_id=62CKV27EUMN7CNMK5TRREU2ERFCNFBW6SNCBEX4DAK5VEAZTOVDQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.901061569 (123h58m10.596910322s) [noise=true , address=185.167.204.206:4064 , piece_id=2N4X3XB5P7BWYZTK3OXOSUK4QR5JKFVYRDML3HCU3DD7VUCXA23A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.910914601 (123h58m10.606763344s) [noise=true , address=185.167.204.205:4006 , piece_id=EJBBNQXXJGP7WHARV36M27UAB3PB7VFXSF3IOAH3GGO5KZ7TYGCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.911431165 (123h58m10.607279918s) [noise=true , address=185.167.204.205:4028 , piece_id=HDTTIT5ZTQXMBLGPC6OE2UQX6UDCPR2BFQMQMPKJMH6ZX44RQTCQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.916389205 (123h58m10.612237958s) [noise=true , address=89.149.201.147:4021 , piece_id=TYM7UGLBNI6RS6ELY22ZCBQD7JNLRWVLDEH4IF5MTAQ4QVUMAFSA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.936866288 (123h58m10.632715041s) [noise=true , address=185.167.204.205:4026 , piece_id=MZYEV4BC7NMVVVU4HH7IVP7ML5DBCKABUZXXLGZLPZU3Y2TXIHWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.945194596 (123h58m10.641043349s) [noise=true , address=185.167.204.205:4020 , piece_id=VKQLVIPGLYNZRHJOYPAT3CYP4CBFMSDHNTO7DWEKGMQP22D6I4TQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.952453869 (123h58m10.648302622s) [noise=true , address=185.167.204.206:4052 , piece_id=SCT45CT34WCU5I3P2UXAQLBS3ONWY4DUNUGWKR5CCLJSRZINN76A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:38.985344929 (123h58m10.681193672s) [noise=true , address=185.167.204.205:4015 , piece_id=2NJ4YWBQPD3SFEV6VK5L6GBX2GU6IBAJTUX5PMS34ML7M3MA3FXA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.012111198 (123h58m10.707959951s) [noise=true , address=185.167.204.205:4024 , piece_id=376UCGXJ4EABB6XJWS4P7IP2FI4ATDRIMLQM7IMGEPTE2F4VZ5FQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:38:39.714925349 (123h58m11.410774093s) [noise=true , address=185.167.204.205:4014 , piece_id=KS6E5ZJASALO4N4KMHVR7JUYDYCCKIVMHM53P2DBKEQAYYJFPZWQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:35.165057551 (123h59m6.860906294s) [noise=true , address=81.171.14.70:4031 , piece_id=6U4WMUFB3WVRYBOZ35JTD3K5QI7TRTQYTCW5NRX3KBFAOZ3VDYBQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: <nil> Nov 4 15:39:36.392079989 (123h59m8.087928732s) [noise=true , address=81.171.14.70:4027 , piece_id=YTCHMGSET6YGCD6IUL7636HOWG2ZL6PD7JRH7GS5KMEGFN52LDPQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392239923 (123h59m8.088088676s) [noise=true , address=81.171.14.70:4025 , piece_id=B2Z26535QLMLAF2QHA6C2GZIAQDTGBX3ZI5IG7XDHZHULYUXU6NA, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392379309 (123h59m8.088228052s) [noise=true , address=81.171.14.70:4032 , piece_id=M4EQZFPNFPM5JNGU4NM5EENBKAE7J5E4JPANLZPW7RYYLQNOF54A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:39:36.392465373 (123h59m8.088314126s) [noise=true , address=81.171.14.70:4036 , piece_id=RHE6WSXEED5COWOWVCIB3WSUY6IG4INOMEAPRR4WN4D3JGJ2AFTQ, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36948->81.171.14.70:4036: use of closed network connection Nov 4 15:39:36.392504658 (123h59m8.088353411s) [noise=true , address=81.171.14.70:4026 , piece_id=QT7Q2JYWLOUV2SGQBHWJF6AZPD72HRDGZRST4MICZQAPEM6NWM4A, seg_pos={0 0}, upload=109193]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38206->81.171.14.70:4026: use of closed network connection Nov 4 15:39:36.392540616 (123h59m8.088389369s) [seg_pos={0 0}, upload=109193]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:39:36.563514451 (123h59m8.259363205s) [upload=109193]: Done uploading segments... Nov 4 15:39:36.563569557 (123h59m8.25941831s) [upload=109193]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 1m0.174185868s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 6 out of 60 is failed (threshold=3) [XX...X...................................................XXX] duration: 1m0.173768999s trace: https://jaeger.dev.storj.io/trace/489d95bb1ac02432 FAIL (2025-11-04 15:38:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== . Nov 4 15:37:36.646201564 (123h57m8.342050317s) [noise=true , address=81.171.14.70:4045 , piece_id=G5QBUROTRU547K2WRY5HE5FFP5PI25NFSJSZXBZRHDFVLKKNFT2A, seg_pos={0 0}, upload=109180]: Uploading piece... Nov 4 15:37:36.646657371 (123h57m8.342506125s) [noise=true , address=185.167.204.205:4030 , piece_id=WMWPYKPSXS3RQEWC7BGHUXDXHP6NRIJBQ6M6PNXNFAEEPCTVLJHA, seg_pos={0 0}, upload=109180]: Uploading piece... Nov 4 15:37:36.647125422 (123h57m8.342974175s) [noise=true , address=185.167.204.205:4012 , piece_id=3HNSQAJ52IQECHDGMPL3OY3FLEHXZHS4JJ37FV7QFKQ576JCOAXQ, seg_pos={0 0}, upload=109180]: Uploading piece... Nov 4 15:37:36.647251072 (123h57m8.343099825s) [noise=true , address=185.167.204.206:4050 , piece_id=TDZS2LQD5J73PUG4P24JQMS62QCQKEFWNLIGL5FEAALBD5JAO67A, seg_pos={0 0}, upload=109180]: Uploading piece... Nov 4 15:37:36.648504549 (123h57m8.344353312s) [noise=true , address=185.167.204.206:4059 , piece_id=SPODNICEADVEFV2UM7KVFIBABNDA6HUE3QKFHYSFDB3BPTKBC2NA, seg_pos={0 0}, upload=109180]: Uploading piece... Nov 4 15:37:36.649357372 (123h57m8.345206125s) [noise=true , address=185.167.204.206:4053 , piece_id=2XDOSB52K2TF5ZTJS5OSMCOEKJTN2F7GDWN7K2RTTZUN667BJBTA, seg_pos={0 0}, upload=109180]: Uploading piece... Nov 4 15:37:36.650044721 (123h57m8.345893464s) [noise=true , address=89.149.201.147:4009 , piece_id=ZDMTX55UVIM4URLC6GDJJOYMOYOSGGXTAMDKLFCHSLH74NVIKCEA, seg_pos={0 0}, upload=109180]: Uploading piece... Nov 4 15:37:36.650699729 (123h57m8.346548492s) [noise=true , address=185.167.204.206:4057 , piece_id=5AEAMZMRS2BL4WWSGS7SOOW5Q7ISNVSF6GICBO5BBUYSWMM5ZL7Q, seg_pos={0 0}, upload=109180]: Uploading piece... Nov 4 15:37:36.652157455 (123h57m8.348006218s) [noise=true , address=89.149.201.147:4005 , piece_id=DH2KSAJMXCO6B77UCWD7KD3GB7W6XPAV5ZBYQ7YX3DRDD5RPBXHQ, seg_pos={0 0}, upload=109180]: Uploading piece... Nov 4 15:37:36.653490974 (123h57m8.349339727s) [noise=true , address=185.167.204.206:4064 , piece_id=6YP733IWM7R5KG7SXC2FTHRB2EHDERPV7FY476PVRPM6VVEKPDPA, seg_pos={0 0}, upload=109180]: Uploading piece... Nov 4 15:37:36.655046686 (123h57m8.350895429s) [noise=true , address=89.149.201.147:4008 , piece_id=GZOVTK47TJ6JVEJKGJH5FB2LTOCBGKO3NZAUHSTI73N2EEUXSHTQ, seg_pos={0 0}, upload=109180]: Uploading piece... Nov 4 15:37:37.545828982 (123h57m9.241677735s) [upload=109180]: Next returned nil segment Nov 4 15:37:37.545852517 (123h57m9.24170127s) [upload=109180]: Waiting for error group managing segments... Nov 4 15:37:37.952686341 (123h57m9.648535094s) [noise=true , address=89.149.201.147:4019 , piece_id=AN46OYIPNQZ42UEFIJSEYPDUYIRKZCY2DYT3U7TQ2UJCTMSZHSXQ, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:37.953864915 (123h57m9.649713669s) [noise=true , address=89.149.201.147:4006 , piece_id=ZQYHG4IF6FLV7BAHT7OOCHBQM6RCE5VIAVVD7SD2SIRITG5ZOD4A, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:37.971961775 (123h57m9.667810518s) [noise=true , address=89.149.201.147:4020 , piece_id=M2QATGBGDOCGAMJJB6HYB7GQHI2MEJWCO4NWCKQCTQTH56XDDUDA, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:37.979109135 (123h57m9.674957888s) [noise=true , address=89.149.201.147:4010 , piece_id=OARJGQBV74PQ3J6GZM4GVNCUAZA37MQAKTYG4VXDR43KVMAC4TDA, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:37.991717363 (123h57m9.687566116s) [noise=true , address=89.149.201.147:4004 , piece_id=2HK6Q7EYYQJB7AMKUGEWWJYQSFCTCPLJKZ7TKVC7STAX2SB7YTYA, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:37.995344410 (123h57m9.691193163s) [noise=true , address=89.149.201.147:4008 , piece_id=GZOVTK47TJ6JVEJKGJH5FB2LTOCBGKO3NZAUHSTI73N2EEUXSHTQ, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.014809815 (123h57m9.710658568s) [noise=true , address=89.149.201.147:4005 , piece_id=DH2KSAJMXCO6B77UCWD7KD3GB7W6XPAV5ZBYQ7YX3DRDD5RPBXHQ, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.114143775 (123h57m9.809992528s) [noise=true , address=89.149.201.147:4009 , piece_id=ZDMTX55UVIM4URLC6GDJJOYMOYOSGGXTAMDKLFCHSLH74NVIKCEA, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.450337016 (123h57m10.146185769s) [noise=true , address=89.149.201.147:4014 , piece_id=4VS4OLOEWRE6KFHZ5XOSYYZYXUZ7IFQ46LC75VDWGA3W4EFMQ7AQ, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.455500789 (123h57m10.151349542s) [noise=true , address=185.167.204.206:4049 , piece_id=NNNZCSSTSBFP65V2BOXW7TORE6KSLP5D2DRG4K2PTSSKZVLIO7DA, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.478497729 (123h57m10.174346472s) [noise=true , address=185.167.204.206:4057 , piece_id=5AEAMZMRS2BL4WWSGS7SOOW5Q7ISNVSF6GICBO5BBUYSWMM5ZL7Q, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.480179462 (123h57m10.176028215s) [noise=true , address=89.149.201.147:4003 , piece_id=HBDNKJJGPWORENK5TJ4LA5E7BQBUUS4FGLRKIA3DD52VJ6KHSZXA, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.483546723 (123h57m10.179395476s) [noise=true , address=185.167.204.206:4068 , piece_id=MCMFU3ABFQGABAHJFBH6JERM5PTZ75GKVPDXM5C6ANOGHCKYHOZA, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.506335949 (123h57m10.202184702s) [noise=true , address=185.167.204.206:4053 , piece_id=2XDOSB52K2TF5ZTJS5OSMCOEKJTN2F7GDWN7K2RTTZUN667BJBTA, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.506452320 (123h57m10.202301073s) [noise=true , address=185.167.204.205:4010 , piece_id=7OHUFWMWLRJ7YC2M6W2DJ7X7SB4AIUMDE4KRNDJ5PBFN5AYTVPOQ, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.511434517 (123h57m10.20728327s) [noise=true , address=185.167.204.205:4030 , piece_id=WMWPYKPSXS3RQEWC7BGHUXDXHP6NRIJBQ6M6PNXNFAEEPCTVLJHA, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.517679338 (123h57m10.213528091s) [noise=true , address=185.167.204.205:4012 , piece_id=3HNSQAJ52IQECHDGMPL3OY3FLEHXZHS4JJ37FV7QFKQ576JCOAXQ, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.518547591 (123h57m10.214396344s) [noise=true , address=185.167.204.206:4060 , piece_id=6YAJVFVGE3VCB5EUQZ47KD6LKQXTMQG7F5XBT7G7NBQTWB5RPKMQ, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.519972695 (123h57m10.215821448s) [noise=true , address=185.167.204.205:4018 , piece_id=CKM2K7UPACNUEQXBGSD6KLJ62FZHGJ2GMS74SVTYF7B4AC6P2MKQ, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.524031965 (123h57m10.219880708s) [noise=true , address=185.167.204.206:4064 , piece_id=6YP733IWM7R5KG7SXC2FTHRB2EHDERPV7FY476PVRPM6VVEKPDPA, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.625963390 (123h57m10.321812133s) [noise=true , address=185.167.204.205:4007 , piece_id=CSBGXVX3C6GXSTXNIY6P3JKE5RSO32RVQI773VXE7FPUN2NU3U7A, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:38.831914227 (123h57m10.52776298s) [noise=true , address=185.167.204.206:4059 , piece_id=SPODNICEADVEFV2UM7KVFIBABNDA6HUE3QKFHYSFDB3BPTKBC2NA, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:37:39.265766659 (123h57m10.961615402s) [noise=true , address=185.167.204.206:4050 , piece_id=TDZS2LQD5J73PUG4P24JQMS62QCQKEFWNLIGL5FEAALBD5JAO67A, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:38:19.669951091 (123h57m51.365799844s) [noise=true , address=81.171.14.70:4045 , piece_id=G5QBUROTRU547K2WRY5HE5FFP5PI25NFSJSZXBZRHDFVLKKNFT2A, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:38:21.208627033 (123h57m52.904475786s) [noise=true , address=81.171.14.70:4028 , piece_id=BMUHWJSCGDMSPIMFK4AGL3WY4FJKKFVEUGZWPDIXNFLBNTEFFM7Q, seg_pos={0 0}, upload=109180]: Done uploading piece. err: <nil> Nov 4 15:38:35.987414788 (123h58m7.683263541s) [noise=true , address=81.171.14.70:4047 , piece_id=3JDZORFELHVXMS6QEVJZN6SZUIGE7AXOAYCYBPB2IHV2PGHSI7CA, seg_pos={0 0}, upload=109180]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:38:35.987441538 (123h58m7.683290291s) [noise=true , address=81.171.14.70:4027 , piece_id=AWLYMBLQWN6UMYGABWY763TMWGRGWFXI2VQQFQPJEN75N2YG7ZZQ, seg_pos={0 0}, upload=109180]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: write tcp 188.166.225.170:37246->81.171.14.70:4027: use of closed network connection; write tcp 188.166.225.170:37246->81.171.14.70:4027: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37246->81.171.14.70:4027: use of closed network connection Nov 4 15:38:35.987595542 (123h58m7.683444285s) [noise=true , address=81.171.14.70:4041 , piece_id=Y5VUHIVCRLVFORFYZE7SU7ONON6THX5LUFBRR5MSCFTHMKIMQI7Q, seg_pos={0 0}, upload=109180]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:53686->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:53686->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53686->81.171.14.70:4041: use of closed network connection Nov 4 15:38:35.987626361 (123h58m7.683475104s) [seg_pos={0 0}, upload=109180]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 15:38:36.158156811 (123h58m7.854005564s) [upload=109180]: Done uploading segments... Nov 4 15:38:36.158196105 (123h58m7.854044858s) [upload=109180]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 1m0.173786723s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [.XX...X...................................................XX] duration: 1m0.171032019s trace: https://jaeger.dev.storj.io/trace/6d36ea01ecc11ce5 FAIL (2025-11-04 15:37:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== 3HRID7A7A, seg_pos={0 0}, upload=109165]: Uploading piece... Nov 4 15:36:36.230374321 (123h56m7.926223074s) [noise=true , address=185.167.204.206:4049 , piece_id=3I4UPOA27EDOPIUCRZDC6S3OG6AUDO3LKDBH2OSEJ6VK5YGBVATA, seg_pos={0 0}, upload=109165]: Uploading piece... Nov 4 15:36:36.230879662 (123h56m7.926728405s) [noise=true , address=185.167.204.206:4044 , piece_id=AYEPWROBR6YIMLYDDBNKFXVYUX6RJ5VCR2OTUJZ44AJE7EPKDBPQ, seg_pos={0 0}, upload=109165]: Uploading piece... Nov 4 15:36:36.231342144 (123h56m7.927190897s) [noise=true , address=81.171.14.70:4025 , piece_id=AYFNX7P447EVHZ5CC5ZC3UZUCHK5KZODURZY3HCWDZ5PFL74IJGA, seg_pos={0 0}, upload=109165]: Uploading piece... Nov 4 15:36:36.231764518 (123h56m7.927613271s) [noise=true , address=185.167.204.205:4005 , piece_id=PQHAYNAGXU3WAGUHRI62OBUXRTYK5RPEEDFANHH5Z5YVS32PWZLA, seg_pos={0 0}, upload=109165]: Uploading piece... Nov 4 15:36:36.232005888 (123h56m7.927854631s) [noise=true , address=185.167.204.206:4072 , piece_id=YP64ONRGFTJD4ARIGUQSYCNYJPQRWVGVBI7QFVE6NZYARQASAG5A, seg_pos={0 0}, upload=109165]: Uploading piece... Nov 4 15:36:36.232552347 (123h56m7.928401101s) [noise=true , address=185.167.204.205:4007 , piece_id=GR6M3EYIXUIHPSDBETX3Q72XC3JW64KEJRTDWGUSTBUJKX43YISQ, seg_pos={0 0}, upload=109165]: Uploading piece... Nov 4 15:36:36.232998207 (123h56m7.92884695s) [noise=true , address=81.171.14.70:4034 , piece_id=L5FPHEWP3A7NC776NWWIHYWY6W5RCTPC66FCZSDKXGMGYOMVODZA, seg_pos={0 0}, upload=109165]: Uploading piece... Nov 4 15:36:36.233444807 (123h56m7.92929355s) [noise=true , address=81.171.14.70:4033 , piece_id=TUG7UOYG723IEULNJ3FW2DIABT3ZCS7ICBNVTIN6YJWP4OYAU5OQ, seg_pos={0 0}, upload=109165]: Uploading piece... Nov 4 15:36:36.234237918 (123h56m7.930086661s) [noise=true , address=185.167.204.206:4068 , piece_id=FRF7HEP4SMEDHDSSOZNNARHWQNGARFRNIIJ6WC5WTYNQUNRLGU2Q, seg_pos={0 0}, upload=109165]: Uploading piece... Nov 4 15:36:36.234679217 (123h56m7.93052797s) [noise=true , address=89.149.201.147:4003 , piece_id=RVBC4W4ND7UTIOAHTHOQAS3KL7ZLQTPWSL6GKUQSW5NWQ2QXYB7Q, seg_pos={0 0}, upload=109165]: Uploading piece... Nov 4 15:36:37.091705949 (123h56m8.787554693s) [upload=109165]: Next returned nil segment Nov 4 15:36:37.091725336 (123h56m8.787574079s) [upload=109165]: Waiting for error group managing segments... Nov 4 15:36:37.559374643 (123h56m9.255223396s) [noise=true , address=89.149.201.147:4001 , piece_id=RZAPGGW2XMX5TUFPHPBDJO3PPNZUPXJUG6JGCJJZCUO273TTJ6NA, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.013689410 (123h56m9.709538163s) [noise=true , address=89.149.201.147:4003 , piece_id=RVBC4W4ND7UTIOAHTHOQAS3KL7ZLQTPWSL6GKUQSW5NWQ2QXYB7Q, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.017650752 (123h56m9.713499495s) [noise=true , address=185.167.204.206:4049 , piece_id=3I4UPOA27EDOPIUCRZDC6S3OG6AUDO3LKDBH2OSEJ6VK5YGBVATA, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.023656619 (123h56m9.719505372s) [noise=true , address=89.149.201.147:4024 , piece_id=LDQFANPJ6NVLJ753LEONE4LLOJW3SPEQJUNQ6YV5DS4V5OTFXTJA, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.031121392 (123h56m9.726970145s) [noise=true , address=185.167.204.205:4005 , piece_id=PQHAYNAGXU3WAGUHRI62OBUXRTYK5RPEEDFANHH5Z5YVS32PWZLA, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.038198188 (123h56m9.734046942s) [noise=true , address=185.167.204.206:4052 , piece_id=F7IIIWLYAJ3ZLU7HCM7W7MXDCBAW6VF2Q4LQP2LO3JCNYF6XA6PQ, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.040741702 (123h56m9.736590455s) [noise=true , address=185.167.204.206:4069 , piece_id=W5VIJAHINTI7YEI4V7QW5UDWZBPSP6JKYDV6D7U74IJYKHHT6VUQ, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.048791810 (123h56m9.744640553s) [noise=true , address=185.167.204.205:4022 , piece_id=FJCZ6LHI5YRCA37Q7LQL6D7Q4BLIZKDW5W6FVKAK4WXF6SFDOVFQ, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.049364992 (123h56m9.745213735s) [noise=true , address=89.149.201.147:4022 , piece_id=73VHKPFXHW7ECGSS4ABIVBASYVEPRLH6AETVW6Z74NCFRECKBJHQ, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.050561821 (123h56m9.746410574s) [noise=true , address=89.149.201.147:4020 , piece_id=PRH4CGG4ZHT2LDOZL3NETILYN23VIN3P7MTHZBDYOPLUKHZ75Y6A, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.056808976 (123h56m9.752657719s) [noise=true , address=89.149.201.147:4015 , piece_id=NQERKH5EK4EGSNFWDVMTUH6EJKQSTI272ZWLBLH4PZJVODOMMBNQ, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.063057926 (123h56m9.758906669s) [noise=true , address=185.167.204.205:4016 , piece_id=GMN4BWEERBYXFMYFN5AG353UR2ZCA4ZBOULSE63CGMJYAAIVKC2Q, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.063213672 (123h56m9.759062425s) [noise=true , address=185.167.204.206:4068 , piece_id=FRF7HEP4SMEDHDSSOZNNARHWQNGARFRNIIJ6WC5WTYNQUNRLGU2Q, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.063907643 (123h56m9.759756386s) [noise=true , address=89.149.201.147:4002 , piece_id=KQTNCBBSSPQRSIFDFVWDN7GKNNRQEFUZBNLMWKKX4CTCJT4APB2Q, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.089620305 (123h56m9.785469058s) [noise=true , address=185.167.204.206:4041 , piece_id=2ECIYBJBCSOQEKZTUPBBJXQKKXEAUQQHRVT4GURHY6Y3HRID7A7A, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.096841967 (123h56m9.79269071s) [noise=true , address=185.167.204.206:4072 , piece_id=YP64ONRGFTJD4ARIGUQSYCNYJPQRWVGVBI7QFVE6NZYARQASAG5A, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.100290193 (123h56m9.796138946s) [noise=true , address=185.167.204.205:4017 , piece_id=VURDRC2BHPMNY2JBRNYMKI3HMBNIGYF5HVOJPHKHNODUYGJOWCMA, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.112631803 (123h56m9.808480546s) [noise=true , address=185.167.204.206:4044 , piece_id=AYEPWROBR6YIMLYDDBNKFXVYUX6RJ5VCR2OTUJZ44AJE7EPKDBPQ, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.119316371 (123h56m9.815165114s) [noise=true , address=185.167.204.205:4002 , piece_id=WPV2PTIQFQ2C3DY5O6PI7LJ4BAA3NADL67ZWELN53JSQTEJ6DLNQ, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.141249546 (123h56m9.837098289s) [noise=true , address=89.149.201.147:4008 , piece_id=2HZWNS6M3NELDRIKY72WF5E6UJ424NKUNJ6GZ2ST3Q2NTLLIXJPA, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.157760246 (123h56m9.853608989s) [noise=true , address=185.167.204.206:4043 , piece_id=G2YCCJMANKN3NIIGY6UBAEBI5FQVIKFDJOJOZIPFHZNMITOWWXTA, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.168231976 (123h56m9.864080729s) [noise=true , address=185.167.204.205:4033 , piece_id=WFB33V3ZP2SLL5YR7F7GUQ5F6XHZ3XV7P7NDNRF7OQLOIFJQYEEA, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:36:38.177398500 (123h56m9.873247243s) [noise=true , address=185.167.204.205:4007 , piece_id=GR6M3EYIXUIHPSDBETX3Q72XC3JW64KEJRTDWGUSTBUJKX43YISQ, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:37:29.221399831 (123h57m0.917248574s) [noise=true , address=81.171.14.70:4035 , piece_id=BY5S7XAUCKUTJBBTQZDDFJCE4FAM7DH72RDBJRCUU2VVNQ7GH7PQ, seg_pos={0 0}, upload=109165]: Done uploading piece. err: <nil> Nov 4 15:37:35.575602222 (123h57m7.271450995s) [noise=true , address=81.171.14.70:4034 , piece_id=L5FPHEWP3A7NC776NWWIHYWY6W5RCTPC66FCZSDKXGMGYOMVODZA, seg_pos={0 0}, upload=109165]: Done uploading piece. err: ecclient: upload failed (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN, address:81.171.14.70:4034): protocol: write tcp 188.166.225.170:42536->81.171.14.70:4034: use of closed network connection; write tcp 188.166.225.170:42536->81.171.14.70:4034: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42536->81.171.14.70:4034: use of closed network connection Nov 4 15:37:35.575713002 (123h57m7.271561745s) [noise=true , address=81.171.14.70:4047 , piece_id=6345OZEBYEFCQEGBXF6HYPJBGT4A3QUXMICY4BB2W23MONGHMLBQ, seg_pos={0 0}, upload=109165]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:37:35.575840514 (123h57m7.271689268s) [noise=true , address=81.171.14.70:4025 , piece_id=AYFNX7P447EVHZ5CC5ZC3UZUCHK5KZODURZY3HCWDZ5PFL74IJGA, seg_pos={0 0}, upload=109165]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:37:35.575960944 (123h57m7.271809687s) [noise=true , address=81.171.14.70:4033 , piece_id=TUG7UOYG723IEULNJ3FW2DIABT3ZCS7ICBNVTIN6YJWP4OYAU5OQ, seg_pos={0 0}, upload=109165]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:39206->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:39206->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39206->81.171.14.70:4033: use of closed network connection Nov 4 15:37:35.575994709 (123h57m7.271843452s) [seg_pos={0 0}, upload=109165]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 4 15:37:35.744207736 (123h57m7.440056489s) [upload=109165]: Done uploading segments... Nov 4 15:37:35.744248934 (123h57m7.440097687s) [upload=109165]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 1m0.171043801s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [..XX...X...................................................X] duration: 1m0.171069102s trace: https://jaeger.dev.storj.io/trace/6050b926f2999cc5 FAIL (2025-11-04 15:36:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== ZL7FL6MDA, seg_pos={0 0}, upload=109147]: Uploading piece... Nov 4 15:35:35.848393315 (123h55m7.544242069s) [noise=true , address=185.167.204.205:4031 , piece_id=6JLFFCYNU4HFJI5ZIGPF27LVTC2IMFGUF2FKS3Z7GQQLS2GR7NLQ, seg_pos={0 0}, upload=109147]: Uploading piece... Nov 4 15:35:35.849987812 (123h55m7.545836565s) [noise=true , address=81.171.14.70:4036 , piece_id=IOWKHHDTX6HPG37P74BLTMDLPKLO66H4P3X7YCTIQ7SFX6JWZPTA, seg_pos={0 0}, upload=109147]: Uploading piece... Nov 4 15:35:35.851237532 (123h55m7.547086285s) [noise=true , address=185.167.204.205:4014 , piece_id=2TCE4QDFYT3WKFNH5A2QV5K6PEUWH5O7KQ5VOPCLBVMPW2TW6NSA, seg_pos={0 0}, upload=109147]: Uploading piece... Nov 4 15:35:35.852513270 (123h55m7.548362023s) [noise=true , address=185.167.204.205:4022 , piece_id=4HOALRA7V6YFIKPBUOLQGWKQH2R4UI3D5WDH7ME6YGTOTZB3G5JQ, seg_pos={0 0}, upload=109147]: Uploading piece... Nov 4 15:35:35.853279910 (123h55m7.549128653s) [noise=true , address=185.167.204.206:4069 , piece_id=DH2I544D7GOJFXLQAMJPM6IPVSDFPYJITPNA6OX4SJFKYRMOG75Q, seg_pos={0 0}, upload=109147]: Uploading piece... Nov 4 15:35:35.854294222 (123h55m7.550142965s) [noise=true , address=81.171.14.70:4044 , piece_id=4XHLS6PQLODMUJNCVKWAC6M5UQOTWN6N4XHZGZ3BPFCWICVBIZAA, seg_pos={0 0}, upload=109147]: Uploading piece... Nov 4 15:35:35.854852595 (123h55m7.550701339s) [noise=true , address=81.171.14.70:4047 , piece_id=54NAAT4CKW4QYXQCJ3K7MIH76HTMET5AKPZKXAJB7XODFQZSWLUQ, seg_pos={0 0}, upload=109147]: Uploading piece... Nov 4 15:35:35.855325776 (123h55m7.551174519s) [noise=true , address=81.171.14.70:4028 , piece_id=XJXEOOC7TTYETYB2CLK2LPPRKOUDB6OSZGKADFYU6OIR7TY3VXMQ, seg_pos={0 0}, upload=109147]: Uploading piece... Nov 4 15:35:36.705860184 (123h55m8.401708937s) [upload=109147]: Next returned nil segment Nov 4 15:35:36.705879481 (123h55m8.401728234s) [upload=109147]: Waiting for error group managing segments... Nov 4 15:35:37.088644433 (123h55m8.784493186s) [noise=true , address=89.149.201.147:4008 , piece_id=BIV7JEBSG2EYFALNR26WCX5PTTERIGEDQQUEFLN4EYFZVOYHJYOQ, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:37.134687578 (123h55m8.830536331s) [noise=true , address=89.149.201.147:4024 , piece_id=QXNGJM4L2PLSXVRODYWTNSRWWJ22T27A5HMCHZTPORMZL7FL6MDA, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:37.144518087 (123h55m8.84036684s) [noise=true , address=89.149.201.147:4009 , piece_id=5XOL2XLASIAPGEJCC24H5ZBFSCVLMO7I7TFUCXEK4IHX2GACHYHA, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:37.147475128 (123h55m8.843323881s) [noise=true , address=89.149.201.147:4015 , piece_id=JGWKJSD2M5CE66LPCHIKUILW6UGNU5DRV7OCKGRSDLER6SJJA5BA, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:37.168223507 (123h55m8.86407228s) [noise=true , address=89.149.201.147:4020 , piece_id=5HH6HHC6CVT6PE6DMX6STDUZKHYCOF4G6VXV25CXPMJ4P4HXIKLQ, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:37.186481723 (123h55m8.882330476s) [noise=true , address=89.149.201.147:4016 , piece_id=4YPJ3MM242OELVWO7ZNVWSXLFFDLLNLWRKR4OXWSMJPVPMHIQHMQ, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:37.226901580 (123h55m8.922750323s) [noise=true , address=89.149.201.147:4002 , piece_id=UYEKTBUVZSMLDHFBCY4T4GTQU2IT6GU33N3SXXRARW7QJ34SVRAA, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:37.259481278 (123h55m8.955330031s) [noise=true , address=89.149.201.147:4004 , piece_id=7ZZYEJIMQZYPSUV5DD42KVNCZUKNSS46AJ6UWVM55IHNEJLXARSQ, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:37.679285419 (123h55m9.375134163s) [noise=true , address=185.167.204.206:4048 , piece_id=23AMNLU5TJCV2QD76S53NMQ6D3DACPPIIVVG5COKSE3FPLRXORQA, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:37.728950239 (123h55m9.424798982s) [noise=true , address=185.167.204.206:4067 , piece_id=TOAHGA2UT4MM3NFKU354U32QKLC3X2PPSYM3Z3XFNHVPZYFLQH2A, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:37.729694647 (123h55m9.42554339s) [noise=true , address=185.167.204.205:4022 , piece_id=4HOALRA7V6YFIKPBUOLQGWKQH2R4UI3D5WDH7ME6YGTOTZB3G5JQ, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:37.730122271 (123h55m9.425971014s) [noise=true , address=185.167.204.205:4031 , piece_id=6JLFFCYNU4HFJI5ZIGPF27LVTC2IMFGUF2FKS3Z7GQQLS2GR7NLQ, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:37.746065780 (123h55m9.441914523s) [noise=true , address=185.167.204.206:4041 , piece_id=CRLVNIXYBRBZAPJ5IM3RDQ4UY6E5RT7KVL3WSIBHDWJCE5HRGWIA, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:37.756949255 (123h55m9.452798008s) [noise=true , address=185.167.204.206:4069 , piece_id=DH2I544D7GOJFXLQAMJPM6IPVSDFPYJITPNA6OX4SJFKYRMOG75Q, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:37.787016138 (123h55m9.482864881s) [noise=true , address=185.167.204.205:4012 , piece_id=QBL6E22KE23CS25ZS5WOFAR75RFBNEFODZSELGBSPOIVSEDHLJVA, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:38.311418242 (123h55m10.007266995s) [noise=true , address=185.167.204.205:4021 , piece_id=GVC6PIVGD5KA6WRIGR7ZO5S2U2ZWTATMH3NJXJMIIBJGWSBAUFNQ, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:38.321889862 (123h55m10.017738615s) [noise=true , address=185.167.204.205:4020 , piece_id=SVZWB2ECQKX3GIJJQAAD7I3CXULCIJEHBMAIJGX2AMVQRMTDJW3A, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:38.336618959 (123h55m10.032467702s) [noise=true , address=185.167.204.205:4014 , piece_id=2TCE4QDFYT3WKFNH5A2QV5K6PEUWH5O7KQ5VOPCLBVMPW2TW6NSA, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:38.337349530 (123h55m10.033198283s) [noise=true , address=185.167.204.206:4071 , piece_id=SSROQRY5D4YUS47HSPOUB45GW6M7K5NAD4I2RGN5YWCRBPIGZSHQ, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:38.382109711 (123h55m10.077958464s) [noise=true , address=185.167.204.205:4016 , piece_id=R6ECQUCNO4HVRH3HVFKGCOTYTIQPDZ4WF33ICVPNHHSZMEHJDC2A, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:52.981054553 (123h55m24.676903306s) [noise=true , address=81.171.14.70:4036 , piece_id=IOWKHHDTX6HPG37P74BLTMDLPKLO66H4P3X7YCTIQ7SFX6JWZPTA, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:35:54.915857531 (123h55m26.611706274s) [noise=true , address=81.171.14.70:4047 , piece_id=54NAAT4CKW4QYXQCJ3K7MIH76HTMET5AKPZKXAJB7XODFQZSWLUQ, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:36:08.692696239 (123h55m40.388544982s) [noise=true , address=81.171.14.70:4027 , piece_id=EV6FWQJ34Q46OLH46NELV4SWCXYNFS56JAWJYRRLE77XHMMTEAHA, seg_pos={0 0}, upload=109147]: Done uploading piece. err: <nil> Nov 4 15:36:35.190859550 (123h56m6.886708303s) [noise=true , address=81.171.14.70:4028 , piece_id=XJXEOOC7TTYETYB2CLK2LPPRKOUDB6OSZGKADFYU6OIR7TY3VXMQ, seg_pos={0 0}, upload=109147]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: write tcp 188.166.225.170:56302->81.171.14.70:4028: use of closed network connection; write tcp 188.166.225.170:56302->81.171.14.70:4028: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56302->81.171.14.70:4028: use of closed network connection Nov 4 15:36:35.190934843 (123h56m6.886783596s) [noise=true , address=81.171.14.70:4038 , piece_id=MBTN3TFZNHHBD7JUZI4SMZNKHDVUYG5ATX67KJXMKWTCBKVOYBOQ, seg_pos={0 0}, upload=109147]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:49018->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:49018->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49018->81.171.14.70:4038: use of closed network connection Nov 4 15:36:35.191087604 (123h56m6.886936347s) [noise=true , address=81.171.14.70:4048 , piece_id=GVQCN2KYI2YJPCTS3JHBXJV4AY2FLRAVZQZMGQAQFJNHKIAAE4SA, seg_pos={0 0}, upload=109147]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:36:35.191162356 (123h56m6.887011099s) [noise=true , address=81.171.14.70:4035 , piece_id=62FCYPSKSNVAO2K6XUMG44FEH63GUKD3XVWFFD2NQR2AGQSJMF7A, seg_pos={0 0}, upload=109147]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:35572->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:35572->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35572->81.171.14.70:4035: use of closed network connection Nov 4 15:36:35.191317933 (123h56m6.887166686s) [noise=true , address=81.171.14.70:4044 , piece_id=4XHLS6PQLODMUJNCVKWAC6M5UQOTWN6N4XHZGZ3BPFCWICVBIZAA, seg_pos={0 0}, upload=109147]: Done uploading piece. err: ecclient: upload failed (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr, address:81.171.14.70:4044): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 15:36:35.191352208 (123h56m6.887200961s) [seg_pos={0 0}, upload=109147]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 4 15:36:35.358891141 (123h56m7.054739884s) [upload=109147]: Done uploading segments... Nov 4 15:36:35.358933392 (123h56m7.054782145s) [upload=109147]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 1m0.171093649s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [...XX...X...................................................] duration: 11.453358618s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:35:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 11.453394917s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [....XX...X..................................................] duration: 40.887839907s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:34:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 40.887873281s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [.....XX...X.................................................] duration: 4.743690721s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:33:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.74372706s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [......XX...X................................................] duration: 10.553811441s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:32:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 10.553835517s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [.......XX...X...............................................] duration: 10.058608708s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:31:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 10.058638214s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [........XX...X..............................................] duration: 22.840611996s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:30:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 22.840628658s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [.........XX...X.............................................] duration: 59.544559685s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:30:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 59.544592036s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [..........XX...X............................................] duration: 4.79798006s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:28:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.798008424s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [...........XX...X...........................................] duration: 5.500188015s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:28:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.500218533s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [............XX...X..........................................] duration: 5.454124652s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:27:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.454157917s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [.............XX...X.........................................] duration: 5.923118129s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:27:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.923148657s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [..............XX...X........................................] duration: 5.425752876s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:26:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.425782082s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [...............XX...X.......................................] duration: 4.740277813s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:25:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.740317138s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [................XX...X......................................] duration: 5.399905428s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:24:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.399939012s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 4 out of 60 is failed (threshold=3) [X................XX...X.....................................] duration: 5.499677574s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:24:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.499709895s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [XX................XX...X....................................] duration: 5.959554478s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:23:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.959587892s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [.XX................XX...X...................................] duration: 4.744424443s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:22:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.744533299s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [..XX................XX...X..................................] duration: 5.619399166s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:21:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.619424896s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [...XX................XX...X.................................] duration: 6.035555646s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:21:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 6.035593359s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [....XX................XX...X................................] duration: 5.445004648s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:20:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.445035426s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [.....XX................XX...X...............................] duration: 5.52508213s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:19:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.525110754s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [......XX................XX...X..............................] duration: 4.886058668s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:18:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.886086972s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [.......XX................XX...X.............................] duration: 5.46740958s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:18:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.467457691s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [........XX................XX...X............................] duration: 4.899409687s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:17:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.899434505s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [.........XX................XX...X...........................] duration: 5.437977305s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:17:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.438006421s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [..........XX................XX...X..........................] duration: 5.508028235s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:16:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.508061979s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 5 out of 60 is failed (threshold=3) [...........XX................XX...X.........................] duration: 4.923290675s trace: https://jaeger.dev.storj.io/trace/10d18538b009858c FAIL (2025-11-04 15:15:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== 9247 (123h15m35.325908s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.638941450 (123h15m35.334790203s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.640691532 (123h15m35.336540285s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.641162839 (123h15m35.337011582s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643113674 (123h15m35.338962427s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.643265223 (123h15m35.339113976s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645023951 (123h15m35.340872694s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.645838251 (123h15m35.341687014s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647607250 (123h15m35.343456003s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.647714364 (123h15m35.343563117s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:03.649207657 (123h15m35.345056411s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Uploading piece... Nov 4 14:56:04.513185096 (123h15m36.209033849s) [upload=108576]: Next returned nil segment Nov 4 14:56:04.513203402 (123h15m36.209052155s) [upload=108576]: Waiting for error group managing segments... Nov 4 14:56:04.959843570 (123h15m36.655692323s) [noise=true , address=89.149.201.147:4021 , piece_id=VAJ7U5KDMF4OTDG7TJXO5QGDCDMWXQGN4AP7BJRAERYP2X4NEKIA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.968808771 (123h15m36.664657514s) [noise=true , address=89.149.201.147:4003 , piece_id=K2SYSXD3HBKQPCATTT27ACJI54ZTJY4TL2A5CCH2FVW5V72RN66A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.969759351 (123h15m36.665608094s) [noise=true , address=89.149.201.147:4017 , piece_id=RFWXZR3RLDN4KOBOKX3Q72SNAC5Z4UBW5WL7QIVSSIIIUPQU75TA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.977103094 (123h15m36.672951838s) [noise=true , address=89.149.201.147:4023 , piece_id=U5UDUXTZMVHNUHNGF7IRGO3HAIRMOWQS5V2Y7DSPBSDYERPRG7LQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.982917807 (123h15m36.67876657s) [noise=true , address=89.149.201.147:4012 , piece_id=KMN6OECIQFXFOOKQSZKFF4AQDFWDQN7PI7M756I4HFDXUFW2G5ZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:04.983319773 (123h15m36.679168526s) [noise=true , address=89.149.201.147:4016 , piece_id=GNQKFQRHSAHXHKU3P3QWUBL5NM4GCLWX2TA5J2ZNYHOULX2M4DEA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.010555694 (123h15m36.706404437s) [noise=true , address=89.149.201.147:4007 , piece_id=LKJTVE6WMPFU4MIBEU7QZDWISIGHICTOIBAZM3TA533NRE7JS6SQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.433190783 (123h15m37.129039536s) [noise=true , address=89.149.201.147:4013 , piece_id=5NH72FZMNMVKVY3AXTCMLRJ5VMBWACIVN2CU5A7JBDLDLWGZQEZA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.444690811 (123h15m37.140539564s) [noise=true , address=185.167.204.205:4022 , piece_id=S77SAQ3YARREE4FTEG4JIROWETWMS3BP6F4L4YTDMHISI7MSFLKA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.471495051 (123h15m37.167343794s) [noise=true , address=185.167.204.206:4068 , piece_id=5IIZ5MSUKYJDPBTTFD42J7MKF6K2UE3DGL47NFLQLSSZWULNFQLA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.484370769 (123h15m37.180219552s) [noise=true , address=185.167.204.206:4038 , piece_id=3HAKKQLLW25T5KOXL7QFGR7RRKJ7NQJFFUQRRLLNX7IBB3E2N3JQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.499302702 (123h15m37.195151475s) [noise=true , address=185.167.204.206:4060 , piece_id=6ABVXJOZWC6ZCRBLCNH75JZXI6C5HG73J3RMVCPPUGOB5XPT52GA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.503539699 (123h15m37.199388482s) [noise=true , address=185.167.204.206:4040 , piece_id=5HO5NXJI3LMWIOVP4JO4IRCTASRPQYM42DVT3KGR4BX2DHHWJS3Q, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.520000463 (123h15m37.215849216s) [noise=true , address=185.167.204.206:4071 , piece_id=52WYUQGVSDU5REPKM5ULMDOOX2WEUO6QQ4MTCOCPSMORMVBIKW3A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.536349534 (123h15m37.232198277s) [noise=true , address=185.167.204.205:4036 , piece_id=63NBXG3LRMXSFK6EKOPIHKXM6BU6M3JDIAWS43XT5SX3SJAVY4KQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.584600884 (123h15m37.280449627s) [noise=true , address=185.167.204.205:4021 , piece_id=ZLPKV6MY2UE7YXKY642T7JKTUSJVQS4RZ6FKBSGTLVQFGGHXWSJA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:05.606823469 (123h15m37.302672222s) [noise=true , address=185.167.204.206:4052 , piece_id=3MFUHHTQBULT2LADCSINEYFLHSCSNC4QP6DHWXCJMYRONUZUZ4CA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.246044654 (123h15m37.941893407s) [noise=true , address=185.167.204.206:4067 , piece_id=5FLBFY4IUNLFV25LWABB63I2ZNJOZ6P3UMDRGYMB2YJSQRDXPLTQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.260480422 (123h15m37.956329175s) [noise=true , address=185.167.204.206:4037 , piece_id=LXGG3I7M3V2I6B5YGCH4WHLO2L6HHPNQJMA2O6K55UW3GFGT3XOQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.262692624 (123h15m37.958541377s) [noise=true , address=185.167.204.206:4064 , piece_id=SDXAZZV4UGL76ZMLJTC64HNNZEMEHK3SWG3TRPBBDEGZ3J5JKMIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.296841890 (123h15m37.992690643s) [noise=true , address=185.167.204.206:4059 , piece_id=5RWXLZR5CH5KOTPTTLZW757E53HFPODE5IFR6IPMOLWB5JTY6JBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:06.309414871 (123h15m38.005263624s) [noise=true , address=185.167.204.206:4058 , piece_id=Y5LOPLAV6BKHTHTYWRDD46ULFILQ2FNOCDIP5GZA5SFYPDXHBHBA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:13.766511664 (123h15m45.462360437s) [noise=true , address=81.171.14.70:4028 , piece_id=ZVJVEJKC7ANLFIGVHB4TFGTDMKKYNS6EQKYXV4CFB5A7OKMXWSOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:14.935899675 (123h15m46.631748418s) [noise=true , address=81.171.14.70:4047 , piece_id=TKWM6IRCR6DTKZNDEV7U7MLZ7OHYRROXWBQOYGHWYVPB2RAUNYXA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:56:30.839956608 (123h16m2.535805361s) [noise=true , address=81.171.14.70:4043 , piece_id=NVIH2K5IVY7IMCPJOYURMHDK7A7V4VDFW5SV2AANHVXOQZ3UAYXQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: <nil> Nov 4 14:57:02.915700057 (123h16m34.61154881s) [noise=true , address=81.171.14.70:4039 , piece_id=J4J7A5OCAX2G6Z3552TI7KOCUKKLLL4VVIVLCP4XKTR7KW73DP6A, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58032->81.171.14.70:4039: use of closed network connection Nov 4 14:57:02.915878617 (123h16m34.61172737s) [noise=true , address=81.171.14.70:4035 , piece_id=E65G5S4SM7TPJL3ZN5DOBE7YAOS2OVYW4BEWZPQPRATLNB4APDOA, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: expected piece hash; context deadline exceeded; EOF Nov 4 14:57:02.915915076 (123h16m34.611763829s) [noise=true , address=81.171.14.70:4041 , piece_id=MYOUW52I4CKALN5Q3WGSLRTG4V2E4IH5EO52TEBI3DLGKLKKGUIQ, seg_pos={0 0}, upload=108576]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35846->81.171.14.70:4041: use of closed network connection Nov 4 14:57:02.915953210 (123h16m34.611801963s) [seg_pos={0 0}, upload=108576]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 4 14:57:03.086888088 (123h16m34.782736841s) [upload=108576]: Done uploading segments... Nov 4 14:57:03.086943063 (123h16m34.782791826s) [upload=108576]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.923320361s core uplink eu-select-1 eu1-placement-15

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)