Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 4.826342173s FAIL (2025-09-26 02:35:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 13) ======LOGS====== gment Sep 26 02:27:20.647600085 (60h56m6.109692568s) [upload=53621]: Waiting for error group managing segments... Sep 26 02:27:21.064033182 (60h56m6.526125665s) [noise=true , address=81.171.14.70:4038 , piece_id=AJ3Q3UP6ML4UPYVLM3VPRMNXULF4P5I57UP5RWBWA66QYKAESHEA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.066538182 (60h56m6.528630665s) [noise=true , address=89.149.201.147:4019 , piece_id=HYDF7GUBYX2ZRSSPNCRKIFDST4FB6WOHEAPYHPMXUYXI2OKCZ5EA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.068616490 (60h56m6.530708963s) [noise=true , address=81.171.14.70:4033 , piece_id=MVFXRRZN43QDVZKTVD3NHO77FDISNOALQUY4UMKMFZVNL67DAIMA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.075913866 (60h56m6.538006349s) [noise=true , address=81.171.14.70:4042 , piece_id=QCOZHOPJT52L5JHTCVPMYISO2UJMBZXXKZXMNPCX4XSJZBGGIF2A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.076377048 (60h56m6.538469531s) [noise=true , address=89.149.201.147:4009 , piece_id=F4MNYJK2MEMWWDKXCZHKVMJSYCC7CVRCHNF4TP2YEOYRQ3K7HFQA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.084005253 (60h56m6.546097747s) [noise=true , address=81.171.14.70:4027 , piece_id=TGOV2RGCAPVKOKDEPXHOWCJ4QFVMHSINAZ2DYUVE5M656SAHIHOA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.094698566 (60h56m6.556791049s) [noise=true , address=89.149.201.147:4021 , piece_id=BDPAYURJ6Z7MGVPK3CR76HZDJ32XZWQHO4WRX63G3GVNUVRMTPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.123121630 (60h56m6.585214103s) [noise=true , address=89.149.201.147:4003 , piece_id=NCSA6HS2G7MLPGYB4AV6IFPQ4ZIJ4UXYPMOL34K47MOH2QHGOVHA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.139178076 (60h56m6.601270559s) [noise=true , address=89.149.201.147:4005 , piece_id=MUOSQMC2MT4VMHRNWYVEEKSETX77CPJJWPXPSUAMXCQJ6XNGWETA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.145390767 (60h56m6.60748325s) [noise=true , address=89.149.201.147:4001 , piece_id=3JQUPRAOJWKUE4YFNW6JXO2TBEHDDESRNS722NFI2P32KEP7FXEQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.153456395 (60h56m6.615548878s) [noise=true , address=89.149.201.147:4023 , piece_id=YQ5PBWVNU3O5K4BSWOR2L63PWP5MON4L3RKVM3NGZAYO2SYHGJYQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.309249281 (60h56m6.771341755s) [noise=true , address=89.149.201.147:4013 , piece_id=OQ5RI7MLAC4A7STBD2IRCNKHUJT3CCKHXTEC34IMZUHAH2TJRPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.557152347 (60h56m7.01924483s) [noise=true , address=81.171.14.70:4043 , piece_id=M4QO4GG7R5PIIK6XG64UNYA6KYWVCBJLIE4Y2YVIE7ORL4BWTXRQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:28:19.148586872 (60h57m4.610679355s) [noise=true , address=185.167.204.205:4016 , piece_id=BBVFJ5TUAQY3HGVKXMAFZTSAVLSHIJD2PKXGGEWTM4RAYWAXFKJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148765031 (60h57m4.610857514s) [noise=true , address=185.167.204.206:4055 , piece_id=IQ62FJCAM7WOQPF7YHG3RQVQK6ZMBHSW6QZXPY3LVODJXSOF6GWQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148791893 (60h57m4.610884366s) [noise=true , address=185.167.204.205:4030 , piece_id=CTD4GIPS6R4VYKV3GTHIGFTTNDMBMHKSB7OYENJ6BSYVBBSOR77A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: rpc: dial tcp 185.167.204.205:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148927972 (60h57m4.611020465s) [noise=true , address=185.167.204.206:4065 , piece_id=AEO6P7V724TTNBNP2D5LRKSXQSZJDGFOXF7MQKDGJXDE3ESK32RQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oPo26KZGZr7o7M18KZiX5PMAAMwHK8FPT59y1hSLhkB6z3Lcj, address:185.167.204.206:4065): protocol: rpc: dial tcp 185.167.204.206:4065: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148936478 (60h57m4.611028961s) [noise=true , address=185.167.204.205:4007 , piece_id=43YBEPUGDIG6G2N2X6WORSU3U442H63OQZK34L7CVS6CK7ZS3FIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: rpc: dial tcp 185.167.204.205:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149093386 (60h57m4.611185869s) [noise=true , address=185.167.204.206:4056 , piece_id=CGFZ3N2VWSSINLTQGUXQTKD3F2RCHBVAAIJ6UHKHLMWDGFTN4SIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, address:185.167.204.206:4056): protocol: rpc: dial tcp 185.167.204.206:4056: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149109788 (60h57m4.611202281s) [noise=true , address=185.167.204.205:4010 , piece_id=7HM6R5VGM6I5CBYDWEB5QAFTMVQODXAGNGO44XKPWWC6V2BF4LZA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: rpc: dial tcp 185.167.204.205:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149259012 (60h57m4.611351505s) [noise=true , address=185.167.204.206:4059 , piece_id=FU7GCK4MIZLDAKCCPXM4QBPE5IHQILHOOJ4XI2T7ZBL3V35NCCMQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149416773 (60h57m4.611509256s) [noise=true , address=185.167.204.206:4062 , piece_id=GCY6UTKPG75JLO36FPZDMFTZWXKL5KEMU2RNB6JBU6VBX7DN3NBQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: rpc: dial tcp 185.167.204.206:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149468791 (60h57m4.611561264s) [noise=true , address=185.167.204.206:4068 , piece_id=MSUITRFBTAFTAOF7SPTQQ7FL3CTFPDGAQG4EEHMXELSCHFWT6WJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149493078 (60h57m4.611585561s) [noise=true , address=185.167.204.206:4043 , piece_id=XTDOMR7BANJPW4PCBMFGLNXRNRQCZYXAWRWOLPLNSF3KK7ONLCKQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: rpc: dial tcp 185.167.204.206:4043: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149500752 (60h57m4.611593235s) [noise=true , address=185.167.204.205:4018 , piece_id=F5OONAIPFWEWJLIZ4UTY5T6526TWHRGWRMSZPHHBYOHCOU6NYPQQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149619167 (60h57m4.61171164s) [noise=true , address=185.167.204.205:4013 , piece_id=6JMD63X4E72LNIZZ55MZ47WWWN2SWRRQNP75C3PZI2CPPVYMLAGA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 188.166.225.170:52378->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149677208 (60h57m4.611769701s) [noise=true , address=185.167.204.206:4054 , piece_id=KLRRYM3F564DQ3BILX33ZBAQNS4OWCIDXKQ7YCBPBGGADJ3R5JLA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u, address:185.167.204.206:4054): protocol: write tcp 188.166.225.170:32858->185.167.204.206:4054: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149731772 (60h57m4.611824245s) [noise=true , address=185.167.204.206:4071 , piece_id=FZ666R6LTM2B2ZSDLLP7XB3NDHMCVABOKVRMUXOHHERM2JU7FU7Q, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149757931 (60h57m4.611850404s) [seg_pos={0 0}, upload=53621]: Done waiting for segment. successful: 13 optimal: 26 errs: <nil> Sep 26 02:28:19.318741697 (60h57m4.78083418s) [upload=53621]: Done uploading segments... Sep 26 02:28:19.318776353 (60h57m4.780868836s) [upload=53621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 13) Duration: 4.826376427s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 4.820435728s FAIL (2025-09-26 02:34:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 13) ======LOGS====== gment Sep 26 02:27:20.647600085 (60h56m6.109692568s) [upload=53621]: Waiting for error group managing segments... Sep 26 02:27:21.064033182 (60h56m6.526125665s) [noise=true , address=81.171.14.70:4038 , piece_id=AJ3Q3UP6ML4UPYVLM3VPRMNXULF4P5I57UP5RWBWA66QYKAESHEA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.066538182 (60h56m6.528630665s) [noise=true , address=89.149.201.147:4019 , piece_id=HYDF7GUBYX2ZRSSPNCRKIFDST4FB6WOHEAPYHPMXUYXI2OKCZ5EA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.068616490 (60h56m6.530708963s) [noise=true , address=81.171.14.70:4033 , piece_id=MVFXRRZN43QDVZKTVD3NHO77FDISNOALQUY4UMKMFZVNL67DAIMA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.075913866 (60h56m6.538006349s) [noise=true , address=81.171.14.70:4042 , piece_id=QCOZHOPJT52L5JHTCVPMYISO2UJMBZXXKZXMNPCX4XSJZBGGIF2A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.076377048 (60h56m6.538469531s) [noise=true , address=89.149.201.147:4009 , piece_id=F4MNYJK2MEMWWDKXCZHKVMJSYCC7CVRCHNF4TP2YEOYRQ3K7HFQA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.084005253 (60h56m6.546097747s) [noise=true , address=81.171.14.70:4027 , piece_id=TGOV2RGCAPVKOKDEPXHOWCJ4QFVMHSINAZ2DYUVE5M656SAHIHOA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.094698566 (60h56m6.556791049s) [noise=true , address=89.149.201.147:4021 , piece_id=BDPAYURJ6Z7MGVPK3CR76HZDJ32XZWQHO4WRX63G3GVNUVRMTPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.123121630 (60h56m6.585214103s) [noise=true , address=89.149.201.147:4003 , piece_id=NCSA6HS2G7MLPGYB4AV6IFPQ4ZIJ4UXYPMOL34K47MOH2QHGOVHA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.139178076 (60h56m6.601270559s) [noise=true , address=89.149.201.147:4005 , piece_id=MUOSQMC2MT4VMHRNWYVEEKSETX77CPJJWPXPSUAMXCQJ6XNGWETA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.145390767 (60h56m6.60748325s) [noise=true , address=89.149.201.147:4001 , piece_id=3JQUPRAOJWKUE4YFNW6JXO2TBEHDDESRNS722NFI2P32KEP7FXEQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.153456395 (60h56m6.615548878s) [noise=true , address=89.149.201.147:4023 , piece_id=YQ5PBWVNU3O5K4BSWOR2L63PWP5MON4L3RKVM3NGZAYO2SYHGJYQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.309249281 (60h56m6.771341755s) [noise=true , address=89.149.201.147:4013 , piece_id=OQ5RI7MLAC4A7STBD2IRCNKHUJT3CCKHXTEC34IMZUHAH2TJRPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.557152347 (60h56m7.01924483s) [noise=true , address=81.171.14.70:4043 , piece_id=M4QO4GG7R5PIIK6XG64UNYA6KYWVCBJLIE4Y2YVIE7ORL4BWTXRQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:28:19.148586872 (60h57m4.610679355s) [noise=true , address=185.167.204.205:4016 , piece_id=BBVFJ5TUAQY3HGVKXMAFZTSAVLSHIJD2PKXGGEWTM4RAYWAXFKJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148765031 (60h57m4.610857514s) [noise=true , address=185.167.204.206:4055 , piece_id=IQ62FJCAM7WOQPF7YHG3RQVQK6ZMBHSW6QZXPY3LVODJXSOF6GWQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148791893 (60h57m4.610884366s) [noise=true , address=185.167.204.205:4030 , piece_id=CTD4GIPS6R4VYKV3GTHIGFTTNDMBMHKSB7OYENJ6BSYVBBSOR77A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: rpc: dial tcp 185.167.204.205:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148927972 (60h57m4.611020465s) [noise=true , address=185.167.204.206:4065 , piece_id=AEO6P7V724TTNBNP2D5LRKSXQSZJDGFOXF7MQKDGJXDE3ESK32RQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oPo26KZGZr7o7M18KZiX5PMAAMwHK8FPT59y1hSLhkB6z3Lcj, address:185.167.204.206:4065): protocol: rpc: dial tcp 185.167.204.206:4065: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148936478 (60h57m4.611028961s) [noise=true , address=185.167.204.205:4007 , piece_id=43YBEPUGDIG6G2N2X6WORSU3U442H63OQZK34L7CVS6CK7ZS3FIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: rpc: dial tcp 185.167.204.205:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149093386 (60h57m4.611185869s) [noise=true , address=185.167.204.206:4056 , piece_id=CGFZ3N2VWSSINLTQGUXQTKD3F2RCHBVAAIJ6UHKHLMWDGFTN4SIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, address:185.167.204.206:4056): protocol: rpc: dial tcp 185.167.204.206:4056: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149109788 (60h57m4.611202281s) [noise=true , address=185.167.204.205:4010 , piece_id=7HM6R5VGM6I5CBYDWEB5QAFTMVQODXAGNGO44XKPWWC6V2BF4LZA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: rpc: dial tcp 185.167.204.205:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149259012 (60h57m4.611351505s) [noise=true , address=185.167.204.206:4059 , piece_id=FU7GCK4MIZLDAKCCPXM4QBPE5IHQILHOOJ4XI2T7ZBL3V35NCCMQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149416773 (60h57m4.611509256s) [noise=true , address=185.167.204.206:4062 , piece_id=GCY6UTKPG75JLO36FPZDMFTZWXKL5KEMU2RNB6JBU6VBX7DN3NBQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: rpc: dial tcp 185.167.204.206:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149468791 (60h57m4.611561264s) [noise=true , address=185.167.204.206:4068 , piece_id=MSUITRFBTAFTAOF7SPTQQ7FL3CTFPDGAQG4EEHMXELSCHFWT6WJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149493078 (60h57m4.611585561s) [noise=true , address=185.167.204.206:4043 , piece_id=XTDOMR7BANJPW4PCBMFGLNXRNRQCZYXAWRWOLPLNSF3KK7ONLCKQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: rpc: dial tcp 185.167.204.206:4043: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149500752 (60h57m4.611593235s) [noise=true , address=185.167.204.205:4018 , piece_id=F5OONAIPFWEWJLIZ4UTY5T6526TWHRGWRMSZPHHBYOHCOU6NYPQQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149619167 (60h57m4.61171164s) [noise=true , address=185.167.204.205:4013 , piece_id=6JMD63X4E72LNIZZ55MZ47WWWN2SWRRQNP75C3PZI2CPPVYMLAGA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 188.166.225.170:52378->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149677208 (60h57m4.611769701s) [noise=true , address=185.167.204.206:4054 , piece_id=KLRRYM3F564DQ3BILX33ZBAQNS4OWCIDXKQ7YCBPBGGADJ3R5JLA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u, address:185.167.204.206:4054): protocol: write tcp 188.166.225.170:32858->185.167.204.206:4054: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149731772 (60h57m4.611824245s) [noise=true , address=185.167.204.206:4071 , piece_id=FZ666R6LTM2B2ZSDLLP7XB3NDHMCVABOKVRMUXOHHERM2JU7FU7Q, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149757931 (60h57m4.611850404s) [seg_pos={0 0}, upload=53621]: Done waiting for segment. successful: 13 optimal: 26 errs: <nil> Sep 26 02:28:19.318741697 (60h57m4.78083418s) [upload=53621]: Done uploading segments... Sep 26 02:28:19.318776353 (60h57m4.780868836s) [upload=53621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 13) Duration: 4.820479242s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 4.938327401s FAIL (2025-09-26 02:34:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 13) ======LOGS====== gment Sep 26 02:27:20.647600085 (60h56m6.109692568s) [upload=53621]: Waiting for error group managing segments... Sep 26 02:27:21.064033182 (60h56m6.526125665s) [noise=true , address=81.171.14.70:4038 , piece_id=AJ3Q3UP6ML4UPYVLM3VPRMNXULF4P5I57UP5RWBWA66QYKAESHEA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.066538182 (60h56m6.528630665s) [noise=true , address=89.149.201.147:4019 , piece_id=HYDF7GUBYX2ZRSSPNCRKIFDST4FB6WOHEAPYHPMXUYXI2OKCZ5EA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.068616490 (60h56m6.530708963s) [noise=true , address=81.171.14.70:4033 , piece_id=MVFXRRZN43QDVZKTVD3NHO77FDISNOALQUY4UMKMFZVNL67DAIMA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.075913866 (60h56m6.538006349s) [noise=true , address=81.171.14.70:4042 , piece_id=QCOZHOPJT52L5JHTCVPMYISO2UJMBZXXKZXMNPCX4XSJZBGGIF2A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.076377048 (60h56m6.538469531s) [noise=true , address=89.149.201.147:4009 , piece_id=F4MNYJK2MEMWWDKXCZHKVMJSYCC7CVRCHNF4TP2YEOYRQ3K7HFQA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.084005253 (60h56m6.546097747s) [noise=true , address=81.171.14.70:4027 , piece_id=TGOV2RGCAPVKOKDEPXHOWCJ4QFVMHSINAZ2DYUVE5M656SAHIHOA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.094698566 (60h56m6.556791049s) [noise=true , address=89.149.201.147:4021 , piece_id=BDPAYURJ6Z7MGVPK3CR76HZDJ32XZWQHO4WRX63G3GVNUVRMTPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.123121630 (60h56m6.585214103s) [noise=true , address=89.149.201.147:4003 , piece_id=NCSA6HS2G7MLPGYB4AV6IFPQ4ZIJ4UXYPMOL34K47MOH2QHGOVHA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.139178076 (60h56m6.601270559s) [noise=true , address=89.149.201.147:4005 , piece_id=MUOSQMC2MT4VMHRNWYVEEKSETX77CPJJWPXPSUAMXCQJ6XNGWETA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.145390767 (60h56m6.60748325s) [noise=true , address=89.149.201.147:4001 , piece_id=3JQUPRAOJWKUE4YFNW6JXO2TBEHDDESRNS722NFI2P32KEP7FXEQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.153456395 (60h56m6.615548878s) [noise=true , address=89.149.201.147:4023 , piece_id=YQ5PBWVNU3O5K4BSWOR2L63PWP5MON4L3RKVM3NGZAYO2SYHGJYQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.309249281 (60h56m6.771341755s) [noise=true , address=89.149.201.147:4013 , piece_id=OQ5RI7MLAC4A7STBD2IRCNKHUJT3CCKHXTEC34IMZUHAH2TJRPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.557152347 (60h56m7.01924483s) [noise=true , address=81.171.14.70:4043 , piece_id=M4QO4GG7R5PIIK6XG64UNYA6KYWVCBJLIE4Y2YVIE7ORL4BWTXRQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:28:19.148586872 (60h57m4.610679355s) [noise=true , address=185.167.204.205:4016 , piece_id=BBVFJ5TUAQY3HGVKXMAFZTSAVLSHIJD2PKXGGEWTM4RAYWAXFKJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148765031 (60h57m4.610857514s) [noise=true , address=185.167.204.206:4055 , piece_id=IQ62FJCAM7WOQPF7YHG3RQVQK6ZMBHSW6QZXPY3LVODJXSOF6GWQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148791893 (60h57m4.610884366s) [noise=true , address=185.167.204.205:4030 , piece_id=CTD4GIPS6R4VYKV3GTHIGFTTNDMBMHKSB7OYENJ6BSYVBBSOR77A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: rpc: dial tcp 185.167.204.205:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148927972 (60h57m4.611020465s) [noise=true , address=185.167.204.206:4065 , piece_id=AEO6P7V724TTNBNP2D5LRKSXQSZJDGFOXF7MQKDGJXDE3ESK32RQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oPo26KZGZr7o7M18KZiX5PMAAMwHK8FPT59y1hSLhkB6z3Lcj, address:185.167.204.206:4065): protocol: rpc: dial tcp 185.167.204.206:4065: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148936478 (60h57m4.611028961s) [noise=true , address=185.167.204.205:4007 , piece_id=43YBEPUGDIG6G2N2X6WORSU3U442H63OQZK34L7CVS6CK7ZS3FIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: rpc: dial tcp 185.167.204.205:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149093386 (60h57m4.611185869s) [noise=true , address=185.167.204.206:4056 , piece_id=CGFZ3N2VWSSINLTQGUXQTKD3F2RCHBVAAIJ6UHKHLMWDGFTN4SIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, address:185.167.204.206:4056): protocol: rpc: dial tcp 185.167.204.206:4056: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149109788 (60h57m4.611202281s) [noise=true , address=185.167.204.205:4010 , piece_id=7HM6R5VGM6I5CBYDWEB5QAFTMVQODXAGNGO44XKPWWC6V2BF4LZA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: rpc: dial tcp 185.167.204.205:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149259012 (60h57m4.611351505s) [noise=true , address=185.167.204.206:4059 , piece_id=FU7GCK4MIZLDAKCCPXM4QBPE5IHQILHOOJ4XI2T7ZBL3V35NCCMQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149416773 (60h57m4.611509256s) [noise=true , address=185.167.204.206:4062 , piece_id=GCY6UTKPG75JLO36FPZDMFTZWXKL5KEMU2RNB6JBU6VBX7DN3NBQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: rpc: dial tcp 185.167.204.206:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149468791 (60h57m4.611561264s) [noise=true , address=185.167.204.206:4068 , piece_id=MSUITRFBTAFTAOF7SPTQQ7FL3CTFPDGAQG4EEHMXELSCHFWT6WJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149493078 (60h57m4.611585561s) [noise=true , address=185.167.204.206:4043 , piece_id=XTDOMR7BANJPW4PCBMFGLNXRNRQCZYXAWRWOLPLNSF3KK7ONLCKQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: rpc: dial tcp 185.167.204.206:4043: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149500752 (60h57m4.611593235s) [noise=true , address=185.167.204.205:4018 , piece_id=F5OONAIPFWEWJLIZ4UTY5T6526TWHRGWRMSZPHHBYOHCOU6NYPQQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149619167 (60h57m4.61171164s) [noise=true , address=185.167.204.205:4013 , piece_id=6JMD63X4E72LNIZZ55MZ47WWWN2SWRRQNP75C3PZI2CPPVYMLAGA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 188.166.225.170:52378->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149677208 (60h57m4.611769701s) [noise=true , address=185.167.204.206:4054 , piece_id=KLRRYM3F564DQ3BILX33ZBAQNS4OWCIDXKQ7YCBPBGGADJ3R5JLA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u, address:185.167.204.206:4054): protocol: write tcp 188.166.225.170:32858->185.167.204.206:4054: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149731772 (60h57m4.611824245s) [noise=true , address=185.167.204.206:4071 , piece_id=FZ666R6LTM2B2ZSDLLP7XB3NDHMCVABOKVRMUXOHHERM2JU7FU7Q, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149757931 (60h57m4.611850404s) [seg_pos={0 0}, upload=53621]: Done waiting for segment. successful: 13 optimal: 26 errs: <nil> Sep 26 02:28:19.318741697 (60h57m4.78083418s) [upload=53621]: Done uploading segments... Sep 26 02:28:19.318776353 (60h57m4.780868836s) [upload=53621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 13) Duration: 4.938387036s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 4.815518537s FAIL (2025-09-26 02:33:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 13) ======LOGS====== gment Sep 26 02:27:20.647600085 (60h56m6.109692568s) [upload=53621]: Waiting for error group managing segments... Sep 26 02:27:21.064033182 (60h56m6.526125665s) [noise=true , address=81.171.14.70:4038 , piece_id=AJ3Q3UP6ML4UPYVLM3VPRMNXULF4P5I57UP5RWBWA66QYKAESHEA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.066538182 (60h56m6.528630665s) [noise=true , address=89.149.201.147:4019 , piece_id=HYDF7GUBYX2ZRSSPNCRKIFDST4FB6WOHEAPYHPMXUYXI2OKCZ5EA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.068616490 (60h56m6.530708963s) [noise=true , address=81.171.14.70:4033 , piece_id=MVFXRRZN43QDVZKTVD3NHO77FDISNOALQUY4UMKMFZVNL67DAIMA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.075913866 (60h56m6.538006349s) [noise=true , address=81.171.14.70:4042 , piece_id=QCOZHOPJT52L5JHTCVPMYISO2UJMBZXXKZXMNPCX4XSJZBGGIF2A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.076377048 (60h56m6.538469531s) [noise=true , address=89.149.201.147:4009 , piece_id=F4MNYJK2MEMWWDKXCZHKVMJSYCC7CVRCHNF4TP2YEOYRQ3K7HFQA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.084005253 (60h56m6.546097747s) [noise=true , address=81.171.14.70:4027 , piece_id=TGOV2RGCAPVKOKDEPXHOWCJ4QFVMHSINAZ2DYUVE5M656SAHIHOA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.094698566 (60h56m6.556791049s) [noise=true , address=89.149.201.147:4021 , piece_id=BDPAYURJ6Z7MGVPK3CR76HZDJ32XZWQHO4WRX63G3GVNUVRMTPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.123121630 (60h56m6.585214103s) [noise=true , address=89.149.201.147:4003 , piece_id=NCSA6HS2G7MLPGYB4AV6IFPQ4ZIJ4UXYPMOL34K47MOH2QHGOVHA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.139178076 (60h56m6.601270559s) [noise=true , address=89.149.201.147:4005 , piece_id=MUOSQMC2MT4VMHRNWYVEEKSETX77CPJJWPXPSUAMXCQJ6XNGWETA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.145390767 (60h56m6.60748325s) [noise=true , address=89.149.201.147:4001 , piece_id=3JQUPRAOJWKUE4YFNW6JXO2TBEHDDESRNS722NFI2P32KEP7FXEQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.153456395 (60h56m6.615548878s) [noise=true , address=89.149.201.147:4023 , piece_id=YQ5PBWVNU3O5K4BSWOR2L63PWP5MON4L3RKVM3NGZAYO2SYHGJYQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.309249281 (60h56m6.771341755s) [noise=true , address=89.149.201.147:4013 , piece_id=OQ5RI7MLAC4A7STBD2IRCNKHUJT3CCKHXTEC34IMZUHAH2TJRPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.557152347 (60h56m7.01924483s) [noise=true , address=81.171.14.70:4043 , piece_id=M4QO4GG7R5PIIK6XG64UNYA6KYWVCBJLIE4Y2YVIE7ORL4BWTXRQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:28:19.148586872 (60h57m4.610679355s) [noise=true , address=185.167.204.205:4016 , piece_id=BBVFJ5TUAQY3HGVKXMAFZTSAVLSHIJD2PKXGGEWTM4RAYWAXFKJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148765031 (60h57m4.610857514s) [noise=true , address=185.167.204.206:4055 , piece_id=IQ62FJCAM7WOQPF7YHG3RQVQK6ZMBHSW6QZXPY3LVODJXSOF6GWQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148791893 (60h57m4.610884366s) [noise=true , address=185.167.204.205:4030 , piece_id=CTD4GIPS6R4VYKV3GTHIGFTTNDMBMHKSB7OYENJ6BSYVBBSOR77A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: rpc: dial tcp 185.167.204.205:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148927972 (60h57m4.611020465s) [noise=true , address=185.167.204.206:4065 , piece_id=AEO6P7V724TTNBNP2D5LRKSXQSZJDGFOXF7MQKDGJXDE3ESK32RQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oPo26KZGZr7o7M18KZiX5PMAAMwHK8FPT59y1hSLhkB6z3Lcj, address:185.167.204.206:4065): protocol: rpc: dial tcp 185.167.204.206:4065: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148936478 (60h57m4.611028961s) [noise=true , address=185.167.204.205:4007 , piece_id=43YBEPUGDIG6G2N2X6WORSU3U442H63OQZK34L7CVS6CK7ZS3FIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: rpc: dial tcp 185.167.204.205:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149093386 (60h57m4.611185869s) [noise=true , address=185.167.204.206:4056 , piece_id=CGFZ3N2VWSSINLTQGUXQTKD3F2RCHBVAAIJ6UHKHLMWDGFTN4SIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, address:185.167.204.206:4056): protocol: rpc: dial tcp 185.167.204.206:4056: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149109788 (60h57m4.611202281s) [noise=true , address=185.167.204.205:4010 , piece_id=7HM6R5VGM6I5CBYDWEB5QAFTMVQODXAGNGO44XKPWWC6V2BF4LZA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: rpc: dial tcp 185.167.204.205:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149259012 (60h57m4.611351505s) [noise=true , address=185.167.204.206:4059 , piece_id=FU7GCK4MIZLDAKCCPXM4QBPE5IHQILHOOJ4XI2T7ZBL3V35NCCMQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149416773 (60h57m4.611509256s) [noise=true , address=185.167.204.206:4062 , piece_id=GCY6UTKPG75JLO36FPZDMFTZWXKL5KEMU2RNB6JBU6VBX7DN3NBQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: rpc: dial tcp 185.167.204.206:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149468791 (60h57m4.611561264s) [noise=true , address=185.167.204.206:4068 , piece_id=MSUITRFBTAFTAOF7SPTQQ7FL3CTFPDGAQG4EEHMXELSCHFWT6WJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149493078 (60h57m4.611585561s) [noise=true , address=185.167.204.206:4043 , piece_id=XTDOMR7BANJPW4PCBMFGLNXRNRQCZYXAWRWOLPLNSF3KK7ONLCKQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: rpc: dial tcp 185.167.204.206:4043: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149500752 (60h57m4.611593235s) [noise=true , address=185.167.204.205:4018 , piece_id=F5OONAIPFWEWJLIZ4UTY5T6526TWHRGWRMSZPHHBYOHCOU6NYPQQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149619167 (60h57m4.61171164s) [noise=true , address=185.167.204.205:4013 , piece_id=6JMD63X4E72LNIZZ55MZ47WWWN2SWRRQNP75C3PZI2CPPVYMLAGA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 188.166.225.170:52378->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149677208 (60h57m4.611769701s) [noise=true , address=185.167.204.206:4054 , piece_id=KLRRYM3F564DQ3BILX33ZBAQNS4OWCIDXKQ7YCBPBGGADJ3R5JLA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u, address:185.167.204.206:4054): protocol: write tcp 188.166.225.170:32858->185.167.204.206:4054: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149731772 (60h57m4.611824245s) [noise=true , address=185.167.204.206:4071 , piece_id=FZ666R6LTM2B2ZSDLLP7XB3NDHMCVABOKVRMUXOHHERM2JU7FU7Q, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149757931 (60h57m4.611850404s) [seg_pos={0 0}, upload=53621]: Done waiting for segment. successful: 13 optimal: 26 errs: <nil> Sep 26 02:28:19.318741697 (60h57m4.78083418s) [upload=53621]: Done uploading segments... Sep 26 02:28:19.318776353 (60h57m4.780868836s) [upload=53621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 13) Duration: 4.815544667s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 4.858503108s FAIL (2025-09-26 02:32:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 13) ======LOGS====== gment Sep 26 02:27:20.647600085 (60h56m6.109692568s) [upload=53621]: Waiting for error group managing segments... Sep 26 02:27:21.064033182 (60h56m6.526125665s) [noise=true , address=81.171.14.70:4038 , piece_id=AJ3Q3UP6ML4UPYVLM3VPRMNXULF4P5I57UP5RWBWA66QYKAESHEA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.066538182 (60h56m6.528630665s) [noise=true , address=89.149.201.147:4019 , piece_id=HYDF7GUBYX2ZRSSPNCRKIFDST4FB6WOHEAPYHPMXUYXI2OKCZ5EA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.068616490 (60h56m6.530708963s) [noise=true , address=81.171.14.70:4033 , piece_id=MVFXRRZN43QDVZKTVD3NHO77FDISNOALQUY4UMKMFZVNL67DAIMA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.075913866 (60h56m6.538006349s) [noise=true , address=81.171.14.70:4042 , piece_id=QCOZHOPJT52L5JHTCVPMYISO2UJMBZXXKZXMNPCX4XSJZBGGIF2A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.076377048 (60h56m6.538469531s) [noise=true , address=89.149.201.147:4009 , piece_id=F4MNYJK2MEMWWDKXCZHKVMJSYCC7CVRCHNF4TP2YEOYRQ3K7HFQA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.084005253 (60h56m6.546097747s) [noise=true , address=81.171.14.70:4027 , piece_id=TGOV2RGCAPVKOKDEPXHOWCJ4QFVMHSINAZ2DYUVE5M656SAHIHOA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.094698566 (60h56m6.556791049s) [noise=true , address=89.149.201.147:4021 , piece_id=BDPAYURJ6Z7MGVPK3CR76HZDJ32XZWQHO4WRX63G3GVNUVRMTPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.123121630 (60h56m6.585214103s) [noise=true , address=89.149.201.147:4003 , piece_id=NCSA6HS2G7MLPGYB4AV6IFPQ4ZIJ4UXYPMOL34K47MOH2QHGOVHA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.139178076 (60h56m6.601270559s) [noise=true , address=89.149.201.147:4005 , piece_id=MUOSQMC2MT4VMHRNWYVEEKSETX77CPJJWPXPSUAMXCQJ6XNGWETA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.145390767 (60h56m6.60748325s) [noise=true , address=89.149.201.147:4001 , piece_id=3JQUPRAOJWKUE4YFNW6JXO2TBEHDDESRNS722NFI2P32KEP7FXEQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.153456395 (60h56m6.615548878s) [noise=true , address=89.149.201.147:4023 , piece_id=YQ5PBWVNU3O5K4BSWOR2L63PWP5MON4L3RKVM3NGZAYO2SYHGJYQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.309249281 (60h56m6.771341755s) [noise=true , address=89.149.201.147:4013 , piece_id=OQ5RI7MLAC4A7STBD2IRCNKHUJT3CCKHXTEC34IMZUHAH2TJRPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.557152347 (60h56m7.01924483s) [noise=true , address=81.171.14.70:4043 , piece_id=M4QO4GG7R5PIIK6XG64UNYA6KYWVCBJLIE4Y2YVIE7ORL4BWTXRQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:28:19.148586872 (60h57m4.610679355s) [noise=true , address=185.167.204.205:4016 , piece_id=BBVFJ5TUAQY3HGVKXMAFZTSAVLSHIJD2PKXGGEWTM4RAYWAXFKJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148765031 (60h57m4.610857514s) [noise=true , address=185.167.204.206:4055 , piece_id=IQ62FJCAM7WOQPF7YHG3RQVQK6ZMBHSW6QZXPY3LVODJXSOF6GWQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148791893 (60h57m4.610884366s) [noise=true , address=185.167.204.205:4030 , piece_id=CTD4GIPS6R4VYKV3GTHIGFTTNDMBMHKSB7OYENJ6BSYVBBSOR77A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: rpc: dial tcp 185.167.204.205:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148927972 (60h57m4.611020465s) [noise=true , address=185.167.204.206:4065 , piece_id=AEO6P7V724TTNBNP2D5LRKSXQSZJDGFOXF7MQKDGJXDE3ESK32RQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oPo26KZGZr7o7M18KZiX5PMAAMwHK8FPT59y1hSLhkB6z3Lcj, address:185.167.204.206:4065): protocol: rpc: dial tcp 185.167.204.206:4065: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148936478 (60h57m4.611028961s) [noise=true , address=185.167.204.205:4007 , piece_id=43YBEPUGDIG6G2N2X6WORSU3U442H63OQZK34L7CVS6CK7ZS3FIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: rpc: dial tcp 185.167.204.205:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149093386 (60h57m4.611185869s) [noise=true , address=185.167.204.206:4056 , piece_id=CGFZ3N2VWSSINLTQGUXQTKD3F2RCHBVAAIJ6UHKHLMWDGFTN4SIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, address:185.167.204.206:4056): protocol: rpc: dial tcp 185.167.204.206:4056: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149109788 (60h57m4.611202281s) [noise=true , address=185.167.204.205:4010 , piece_id=7HM6R5VGM6I5CBYDWEB5QAFTMVQODXAGNGO44XKPWWC6V2BF4LZA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: rpc: dial tcp 185.167.204.205:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149259012 (60h57m4.611351505s) [noise=true , address=185.167.204.206:4059 , piece_id=FU7GCK4MIZLDAKCCPXM4QBPE5IHQILHOOJ4XI2T7ZBL3V35NCCMQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149416773 (60h57m4.611509256s) [noise=true , address=185.167.204.206:4062 , piece_id=GCY6UTKPG75JLO36FPZDMFTZWXKL5KEMU2RNB6JBU6VBX7DN3NBQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: rpc: dial tcp 185.167.204.206:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149468791 (60h57m4.611561264s) [noise=true , address=185.167.204.206:4068 , piece_id=MSUITRFBTAFTAOF7SPTQQ7FL3CTFPDGAQG4EEHMXELSCHFWT6WJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149493078 (60h57m4.611585561s) [noise=true , address=185.167.204.206:4043 , piece_id=XTDOMR7BANJPW4PCBMFGLNXRNRQCZYXAWRWOLPLNSF3KK7ONLCKQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: rpc: dial tcp 185.167.204.206:4043: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149500752 (60h57m4.611593235s) [noise=true , address=185.167.204.205:4018 , piece_id=F5OONAIPFWEWJLIZ4UTY5T6526TWHRGWRMSZPHHBYOHCOU6NYPQQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149619167 (60h57m4.61171164s) [noise=true , address=185.167.204.205:4013 , piece_id=6JMD63X4E72LNIZZ55MZ47WWWN2SWRRQNP75C3PZI2CPPVYMLAGA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 188.166.225.170:52378->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149677208 (60h57m4.611769701s) [noise=true , address=185.167.204.206:4054 , piece_id=KLRRYM3F564DQ3BILX33ZBAQNS4OWCIDXKQ7YCBPBGGADJ3R5JLA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u, address:185.167.204.206:4054): protocol: write tcp 188.166.225.170:32858->185.167.204.206:4054: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149731772 (60h57m4.611824245s) [noise=true , address=185.167.204.206:4071 , piece_id=FZ666R6LTM2B2ZSDLLP7XB3NDHMCVABOKVRMUXOHHERM2JU7FU7Q, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149757931 (60h57m4.611850404s) [seg_pos={0 0}, upload=53621]: Done waiting for segment. successful: 13 optimal: 26 errs: <nil> Sep 26 02:28:19.318741697 (60h57m4.78083418s) [upload=53621]: Done uploading segments... Sep 26 02:28:19.318776353 (60h57m4.780868836s) [upload=53621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 13) Duration: 4.858539026s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 4.918009933s FAIL (2025-09-26 02:31:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 13) ======LOGS====== gment Sep 26 02:27:20.647600085 (60h56m6.109692568s) [upload=53621]: Waiting for error group managing segments... Sep 26 02:27:21.064033182 (60h56m6.526125665s) [noise=true , address=81.171.14.70:4038 , piece_id=AJ3Q3UP6ML4UPYVLM3VPRMNXULF4P5I57UP5RWBWA66QYKAESHEA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.066538182 (60h56m6.528630665s) [noise=true , address=89.149.201.147:4019 , piece_id=HYDF7GUBYX2ZRSSPNCRKIFDST4FB6WOHEAPYHPMXUYXI2OKCZ5EA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.068616490 (60h56m6.530708963s) [noise=true , address=81.171.14.70:4033 , piece_id=MVFXRRZN43QDVZKTVD3NHO77FDISNOALQUY4UMKMFZVNL67DAIMA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.075913866 (60h56m6.538006349s) [noise=true , address=81.171.14.70:4042 , piece_id=QCOZHOPJT52L5JHTCVPMYISO2UJMBZXXKZXMNPCX4XSJZBGGIF2A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.076377048 (60h56m6.538469531s) [noise=true , address=89.149.201.147:4009 , piece_id=F4MNYJK2MEMWWDKXCZHKVMJSYCC7CVRCHNF4TP2YEOYRQ3K7HFQA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.084005253 (60h56m6.546097747s) [noise=true , address=81.171.14.70:4027 , piece_id=TGOV2RGCAPVKOKDEPXHOWCJ4QFVMHSINAZ2DYUVE5M656SAHIHOA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.094698566 (60h56m6.556791049s) [noise=true , address=89.149.201.147:4021 , piece_id=BDPAYURJ6Z7MGVPK3CR76HZDJ32XZWQHO4WRX63G3GVNUVRMTPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.123121630 (60h56m6.585214103s) [noise=true , address=89.149.201.147:4003 , piece_id=NCSA6HS2G7MLPGYB4AV6IFPQ4ZIJ4UXYPMOL34K47MOH2QHGOVHA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.139178076 (60h56m6.601270559s) [noise=true , address=89.149.201.147:4005 , piece_id=MUOSQMC2MT4VMHRNWYVEEKSETX77CPJJWPXPSUAMXCQJ6XNGWETA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.145390767 (60h56m6.60748325s) [noise=true , address=89.149.201.147:4001 , piece_id=3JQUPRAOJWKUE4YFNW6JXO2TBEHDDESRNS722NFI2P32KEP7FXEQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.153456395 (60h56m6.615548878s) [noise=true , address=89.149.201.147:4023 , piece_id=YQ5PBWVNU3O5K4BSWOR2L63PWP5MON4L3RKVM3NGZAYO2SYHGJYQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.309249281 (60h56m6.771341755s) [noise=true , address=89.149.201.147:4013 , piece_id=OQ5RI7MLAC4A7STBD2IRCNKHUJT3CCKHXTEC34IMZUHAH2TJRPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.557152347 (60h56m7.01924483s) [noise=true , address=81.171.14.70:4043 , piece_id=M4QO4GG7R5PIIK6XG64UNYA6KYWVCBJLIE4Y2YVIE7ORL4BWTXRQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:28:19.148586872 (60h57m4.610679355s) [noise=true , address=185.167.204.205:4016 , piece_id=BBVFJ5TUAQY3HGVKXMAFZTSAVLSHIJD2PKXGGEWTM4RAYWAXFKJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148765031 (60h57m4.610857514s) [noise=true , address=185.167.204.206:4055 , piece_id=IQ62FJCAM7WOQPF7YHG3RQVQK6ZMBHSW6QZXPY3LVODJXSOF6GWQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148791893 (60h57m4.610884366s) [noise=true , address=185.167.204.205:4030 , piece_id=CTD4GIPS6R4VYKV3GTHIGFTTNDMBMHKSB7OYENJ6BSYVBBSOR77A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: rpc: dial tcp 185.167.204.205:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148927972 (60h57m4.611020465s) [noise=true , address=185.167.204.206:4065 , piece_id=AEO6P7V724TTNBNP2D5LRKSXQSZJDGFOXF7MQKDGJXDE3ESK32RQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oPo26KZGZr7o7M18KZiX5PMAAMwHK8FPT59y1hSLhkB6z3Lcj, address:185.167.204.206:4065): protocol: rpc: dial tcp 185.167.204.206:4065: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148936478 (60h57m4.611028961s) [noise=true , address=185.167.204.205:4007 , piece_id=43YBEPUGDIG6G2N2X6WORSU3U442H63OQZK34L7CVS6CK7ZS3FIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: rpc: dial tcp 185.167.204.205:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149093386 (60h57m4.611185869s) [noise=true , address=185.167.204.206:4056 , piece_id=CGFZ3N2VWSSINLTQGUXQTKD3F2RCHBVAAIJ6UHKHLMWDGFTN4SIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, address:185.167.204.206:4056): protocol: rpc: dial tcp 185.167.204.206:4056: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149109788 (60h57m4.611202281s) [noise=true , address=185.167.204.205:4010 , piece_id=7HM6R5VGM6I5CBYDWEB5QAFTMVQODXAGNGO44XKPWWC6V2BF4LZA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: rpc: dial tcp 185.167.204.205:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149259012 (60h57m4.611351505s) [noise=true , address=185.167.204.206:4059 , piece_id=FU7GCK4MIZLDAKCCPXM4QBPE5IHQILHOOJ4XI2T7ZBL3V35NCCMQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149416773 (60h57m4.611509256s) [noise=true , address=185.167.204.206:4062 , piece_id=GCY6UTKPG75JLO36FPZDMFTZWXKL5KEMU2RNB6JBU6VBX7DN3NBQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: rpc: dial tcp 185.167.204.206:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149468791 (60h57m4.611561264s) [noise=true , address=185.167.204.206:4068 , piece_id=MSUITRFBTAFTAOF7SPTQQ7FL3CTFPDGAQG4EEHMXELSCHFWT6WJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149493078 (60h57m4.611585561s) [noise=true , address=185.167.204.206:4043 , piece_id=XTDOMR7BANJPW4PCBMFGLNXRNRQCZYXAWRWOLPLNSF3KK7ONLCKQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: rpc: dial tcp 185.167.204.206:4043: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149500752 (60h57m4.611593235s) [noise=true , address=185.167.204.205:4018 , piece_id=F5OONAIPFWEWJLIZ4UTY5T6526TWHRGWRMSZPHHBYOHCOU6NYPQQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149619167 (60h57m4.61171164s) [noise=true , address=185.167.204.205:4013 , piece_id=6JMD63X4E72LNIZZ55MZ47WWWN2SWRRQNP75C3PZI2CPPVYMLAGA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 188.166.225.170:52378->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149677208 (60h57m4.611769701s) [noise=true , address=185.167.204.206:4054 , piece_id=KLRRYM3F564DQ3BILX33ZBAQNS4OWCIDXKQ7YCBPBGGADJ3R5JLA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u, address:185.167.204.206:4054): protocol: write tcp 188.166.225.170:32858->185.167.204.206:4054: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149731772 (60h57m4.611824245s) [noise=true , address=185.167.204.206:4071 , piece_id=FZ666R6LTM2B2ZSDLLP7XB3NDHMCVABOKVRMUXOHHERM2JU7FU7Q, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149757931 (60h57m4.611850404s) [seg_pos={0 0}, upload=53621]: Done waiting for segment. successful: 13 optimal: 26 errs: <nil> Sep 26 02:28:19.318741697 (60h57m4.78083418s) [upload=53621]: Done uploading segments... Sep 26 02:28:19.318776353 (60h57m4.780868836s) [upload=53621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 13) Duration: 4.918041173s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 4.867043217s FAIL (2025-09-26 02:30:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 13) ======LOGS====== gment Sep 26 02:27:20.647600085 (60h56m6.109692568s) [upload=53621]: Waiting for error group managing segments... Sep 26 02:27:21.064033182 (60h56m6.526125665s) [noise=true , address=81.171.14.70:4038 , piece_id=AJ3Q3UP6ML4UPYVLM3VPRMNXULF4P5I57UP5RWBWA66QYKAESHEA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.066538182 (60h56m6.528630665s) [noise=true , address=89.149.201.147:4019 , piece_id=HYDF7GUBYX2ZRSSPNCRKIFDST4FB6WOHEAPYHPMXUYXI2OKCZ5EA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.068616490 (60h56m6.530708963s) [noise=true , address=81.171.14.70:4033 , piece_id=MVFXRRZN43QDVZKTVD3NHO77FDISNOALQUY4UMKMFZVNL67DAIMA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.075913866 (60h56m6.538006349s) [noise=true , address=81.171.14.70:4042 , piece_id=QCOZHOPJT52L5JHTCVPMYISO2UJMBZXXKZXMNPCX4XSJZBGGIF2A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.076377048 (60h56m6.538469531s) [noise=true , address=89.149.201.147:4009 , piece_id=F4MNYJK2MEMWWDKXCZHKVMJSYCC7CVRCHNF4TP2YEOYRQ3K7HFQA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.084005253 (60h56m6.546097747s) [noise=true , address=81.171.14.70:4027 , piece_id=TGOV2RGCAPVKOKDEPXHOWCJ4QFVMHSINAZ2DYUVE5M656SAHIHOA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.094698566 (60h56m6.556791049s) [noise=true , address=89.149.201.147:4021 , piece_id=BDPAYURJ6Z7MGVPK3CR76HZDJ32XZWQHO4WRX63G3GVNUVRMTPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.123121630 (60h56m6.585214103s) [noise=true , address=89.149.201.147:4003 , piece_id=NCSA6HS2G7MLPGYB4AV6IFPQ4ZIJ4UXYPMOL34K47MOH2QHGOVHA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.139178076 (60h56m6.601270559s) [noise=true , address=89.149.201.147:4005 , piece_id=MUOSQMC2MT4VMHRNWYVEEKSETX77CPJJWPXPSUAMXCQJ6XNGWETA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.145390767 (60h56m6.60748325s) [noise=true , address=89.149.201.147:4001 , piece_id=3JQUPRAOJWKUE4YFNW6JXO2TBEHDDESRNS722NFI2P32KEP7FXEQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.153456395 (60h56m6.615548878s) [noise=true , address=89.149.201.147:4023 , piece_id=YQ5PBWVNU3O5K4BSWOR2L63PWP5MON4L3RKVM3NGZAYO2SYHGJYQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.309249281 (60h56m6.771341755s) [noise=true , address=89.149.201.147:4013 , piece_id=OQ5RI7MLAC4A7STBD2IRCNKHUJT3CCKHXTEC34IMZUHAH2TJRPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.557152347 (60h56m7.01924483s) [noise=true , address=81.171.14.70:4043 , piece_id=M4QO4GG7R5PIIK6XG64UNYA6KYWVCBJLIE4Y2YVIE7ORL4BWTXRQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:28:19.148586872 (60h57m4.610679355s) [noise=true , address=185.167.204.205:4016 , piece_id=BBVFJ5TUAQY3HGVKXMAFZTSAVLSHIJD2PKXGGEWTM4RAYWAXFKJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148765031 (60h57m4.610857514s) [noise=true , address=185.167.204.206:4055 , piece_id=IQ62FJCAM7WOQPF7YHG3RQVQK6ZMBHSW6QZXPY3LVODJXSOF6GWQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148791893 (60h57m4.610884366s) [noise=true , address=185.167.204.205:4030 , piece_id=CTD4GIPS6R4VYKV3GTHIGFTTNDMBMHKSB7OYENJ6BSYVBBSOR77A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: rpc: dial tcp 185.167.204.205:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148927972 (60h57m4.611020465s) [noise=true , address=185.167.204.206:4065 , piece_id=AEO6P7V724TTNBNP2D5LRKSXQSZJDGFOXF7MQKDGJXDE3ESK32RQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oPo26KZGZr7o7M18KZiX5PMAAMwHK8FPT59y1hSLhkB6z3Lcj, address:185.167.204.206:4065): protocol: rpc: dial tcp 185.167.204.206:4065: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148936478 (60h57m4.611028961s) [noise=true , address=185.167.204.205:4007 , piece_id=43YBEPUGDIG6G2N2X6WORSU3U442H63OQZK34L7CVS6CK7ZS3FIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: rpc: dial tcp 185.167.204.205:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149093386 (60h57m4.611185869s) [noise=true , address=185.167.204.206:4056 , piece_id=CGFZ3N2VWSSINLTQGUXQTKD3F2RCHBVAAIJ6UHKHLMWDGFTN4SIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, address:185.167.204.206:4056): protocol: rpc: dial tcp 185.167.204.206:4056: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149109788 (60h57m4.611202281s) [noise=true , address=185.167.204.205:4010 , piece_id=7HM6R5VGM6I5CBYDWEB5QAFTMVQODXAGNGO44XKPWWC6V2BF4LZA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: rpc: dial tcp 185.167.204.205:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149259012 (60h57m4.611351505s) [noise=true , address=185.167.204.206:4059 , piece_id=FU7GCK4MIZLDAKCCPXM4QBPE5IHQILHOOJ4XI2T7ZBL3V35NCCMQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149416773 (60h57m4.611509256s) [noise=true , address=185.167.204.206:4062 , piece_id=GCY6UTKPG75JLO36FPZDMFTZWXKL5KEMU2RNB6JBU6VBX7DN3NBQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: rpc: dial tcp 185.167.204.206:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149468791 (60h57m4.611561264s) [noise=true , address=185.167.204.206:4068 , piece_id=MSUITRFBTAFTAOF7SPTQQ7FL3CTFPDGAQG4EEHMXELSCHFWT6WJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149493078 (60h57m4.611585561s) [noise=true , address=185.167.204.206:4043 , piece_id=XTDOMR7BANJPW4PCBMFGLNXRNRQCZYXAWRWOLPLNSF3KK7ONLCKQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: rpc: dial tcp 185.167.204.206:4043: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149500752 (60h57m4.611593235s) [noise=true , address=185.167.204.205:4018 , piece_id=F5OONAIPFWEWJLIZ4UTY5T6526TWHRGWRMSZPHHBYOHCOU6NYPQQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149619167 (60h57m4.61171164s) [noise=true , address=185.167.204.205:4013 , piece_id=6JMD63X4E72LNIZZ55MZ47WWWN2SWRRQNP75C3PZI2CPPVYMLAGA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 188.166.225.170:52378->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149677208 (60h57m4.611769701s) [noise=true , address=185.167.204.206:4054 , piece_id=KLRRYM3F564DQ3BILX33ZBAQNS4OWCIDXKQ7YCBPBGGADJ3R5JLA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u, address:185.167.204.206:4054): protocol: write tcp 188.166.225.170:32858->185.167.204.206:4054: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149731772 (60h57m4.611824245s) [noise=true , address=185.167.204.206:4071 , piece_id=FZ666R6LTM2B2ZSDLLP7XB3NDHMCVABOKVRMUXOHHERM2JU7FU7Q, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149757931 (60h57m4.611850404s) [seg_pos={0 0}, upload=53621]: Done waiting for segment. successful: 13 optimal: 26 errs: <nil> Sep 26 02:28:19.318741697 (60h57m4.78083418s) [upload=53621]: Done uploading segments... Sep 26 02:28:19.318776353 (60h57m4.780868836s) [upload=53621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 13) Duration: 4.867076951s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.069762025s FAIL (2025-09-26 02:30:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 13) ======LOGS====== gment Sep 26 02:27:20.647600085 (60h56m6.109692568s) [upload=53621]: Waiting for error group managing segments... Sep 26 02:27:21.064033182 (60h56m6.526125665s) [noise=true , address=81.171.14.70:4038 , piece_id=AJ3Q3UP6ML4UPYVLM3VPRMNXULF4P5I57UP5RWBWA66QYKAESHEA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.066538182 (60h56m6.528630665s) [noise=true , address=89.149.201.147:4019 , piece_id=HYDF7GUBYX2ZRSSPNCRKIFDST4FB6WOHEAPYHPMXUYXI2OKCZ5EA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.068616490 (60h56m6.530708963s) [noise=true , address=81.171.14.70:4033 , piece_id=MVFXRRZN43QDVZKTVD3NHO77FDISNOALQUY4UMKMFZVNL67DAIMA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.075913866 (60h56m6.538006349s) [noise=true , address=81.171.14.70:4042 , piece_id=QCOZHOPJT52L5JHTCVPMYISO2UJMBZXXKZXMNPCX4XSJZBGGIF2A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.076377048 (60h56m6.538469531s) [noise=true , address=89.149.201.147:4009 , piece_id=F4MNYJK2MEMWWDKXCZHKVMJSYCC7CVRCHNF4TP2YEOYRQ3K7HFQA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.084005253 (60h56m6.546097747s) [noise=true , address=81.171.14.70:4027 , piece_id=TGOV2RGCAPVKOKDEPXHOWCJ4QFVMHSINAZ2DYUVE5M656SAHIHOA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.094698566 (60h56m6.556791049s) [noise=true , address=89.149.201.147:4021 , piece_id=BDPAYURJ6Z7MGVPK3CR76HZDJ32XZWQHO4WRX63G3GVNUVRMTPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.123121630 (60h56m6.585214103s) [noise=true , address=89.149.201.147:4003 , piece_id=NCSA6HS2G7MLPGYB4AV6IFPQ4ZIJ4UXYPMOL34K47MOH2QHGOVHA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.139178076 (60h56m6.601270559s) [noise=true , address=89.149.201.147:4005 , piece_id=MUOSQMC2MT4VMHRNWYVEEKSETX77CPJJWPXPSUAMXCQJ6XNGWETA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.145390767 (60h56m6.60748325s) [noise=true , address=89.149.201.147:4001 , piece_id=3JQUPRAOJWKUE4YFNW6JXO2TBEHDDESRNS722NFI2P32KEP7FXEQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.153456395 (60h56m6.615548878s) [noise=true , address=89.149.201.147:4023 , piece_id=YQ5PBWVNU3O5K4BSWOR2L63PWP5MON4L3RKVM3NGZAYO2SYHGJYQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.309249281 (60h56m6.771341755s) [noise=true , address=89.149.201.147:4013 , piece_id=OQ5RI7MLAC4A7STBD2IRCNKHUJT3CCKHXTEC34IMZUHAH2TJRPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.557152347 (60h56m7.01924483s) [noise=true , address=81.171.14.70:4043 , piece_id=M4QO4GG7R5PIIK6XG64UNYA6KYWVCBJLIE4Y2YVIE7ORL4BWTXRQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:28:19.148586872 (60h57m4.610679355s) [noise=true , address=185.167.204.205:4016 , piece_id=BBVFJ5TUAQY3HGVKXMAFZTSAVLSHIJD2PKXGGEWTM4RAYWAXFKJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148765031 (60h57m4.610857514s) [noise=true , address=185.167.204.206:4055 , piece_id=IQ62FJCAM7WOQPF7YHG3RQVQK6ZMBHSW6QZXPY3LVODJXSOF6GWQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148791893 (60h57m4.610884366s) [noise=true , address=185.167.204.205:4030 , piece_id=CTD4GIPS6R4VYKV3GTHIGFTTNDMBMHKSB7OYENJ6BSYVBBSOR77A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: rpc: dial tcp 185.167.204.205:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148927972 (60h57m4.611020465s) [noise=true , address=185.167.204.206:4065 , piece_id=AEO6P7V724TTNBNP2D5LRKSXQSZJDGFOXF7MQKDGJXDE3ESK32RQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oPo26KZGZr7o7M18KZiX5PMAAMwHK8FPT59y1hSLhkB6z3Lcj, address:185.167.204.206:4065): protocol: rpc: dial tcp 185.167.204.206:4065: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148936478 (60h57m4.611028961s) [noise=true , address=185.167.204.205:4007 , piece_id=43YBEPUGDIG6G2N2X6WORSU3U442H63OQZK34L7CVS6CK7ZS3FIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: rpc: dial tcp 185.167.204.205:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149093386 (60h57m4.611185869s) [noise=true , address=185.167.204.206:4056 , piece_id=CGFZ3N2VWSSINLTQGUXQTKD3F2RCHBVAAIJ6UHKHLMWDGFTN4SIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, address:185.167.204.206:4056): protocol: rpc: dial tcp 185.167.204.206:4056: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149109788 (60h57m4.611202281s) [noise=true , address=185.167.204.205:4010 , piece_id=7HM6R5VGM6I5CBYDWEB5QAFTMVQODXAGNGO44XKPWWC6V2BF4LZA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: rpc: dial tcp 185.167.204.205:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149259012 (60h57m4.611351505s) [noise=true , address=185.167.204.206:4059 , piece_id=FU7GCK4MIZLDAKCCPXM4QBPE5IHQILHOOJ4XI2T7ZBL3V35NCCMQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149416773 (60h57m4.611509256s) [noise=true , address=185.167.204.206:4062 , piece_id=GCY6UTKPG75JLO36FPZDMFTZWXKL5KEMU2RNB6JBU6VBX7DN3NBQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: rpc: dial tcp 185.167.204.206:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149468791 (60h57m4.611561264s) [noise=true , address=185.167.204.206:4068 , piece_id=MSUITRFBTAFTAOF7SPTQQ7FL3CTFPDGAQG4EEHMXELSCHFWT6WJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149493078 (60h57m4.611585561s) [noise=true , address=185.167.204.206:4043 , piece_id=XTDOMR7BANJPW4PCBMFGLNXRNRQCZYXAWRWOLPLNSF3KK7ONLCKQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: rpc: dial tcp 185.167.204.206:4043: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149500752 (60h57m4.611593235s) [noise=true , address=185.167.204.205:4018 , piece_id=F5OONAIPFWEWJLIZ4UTY5T6526TWHRGWRMSZPHHBYOHCOU6NYPQQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149619167 (60h57m4.61171164s) [noise=true , address=185.167.204.205:4013 , piece_id=6JMD63X4E72LNIZZ55MZ47WWWN2SWRRQNP75C3PZI2CPPVYMLAGA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 188.166.225.170:52378->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149677208 (60h57m4.611769701s) [noise=true , address=185.167.204.206:4054 , piece_id=KLRRYM3F564DQ3BILX33ZBAQNS4OWCIDXKQ7YCBPBGGADJ3R5JLA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u, address:185.167.204.206:4054): protocol: write tcp 188.166.225.170:32858->185.167.204.206:4054: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149731772 (60h57m4.611824245s) [noise=true , address=185.167.204.206:4071 , piece_id=FZ666R6LTM2B2ZSDLLP7XB3NDHMCVABOKVRMUXOHHERM2JU7FU7Q, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149757931 (60h57m4.611850404s) [seg_pos={0 0}, upload=53621]: Done waiting for segment. successful: 13 optimal: 26 errs: <nil> Sep 26 02:28:19.318741697 (60h57m4.78083418s) [upload=53621]: Done uploading segments... Sep 26 02:28:19.318776353 (60h57m4.780868836s) [upload=53621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 13) Duration: 5.06982223s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 4.84938543s FAIL (2025-09-26 02:29:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 13) ======LOGS====== gment Sep 26 02:27:20.647600085 (60h56m6.109692568s) [upload=53621]: Waiting for error group managing segments... Sep 26 02:27:21.064033182 (60h56m6.526125665s) [noise=true , address=81.171.14.70:4038 , piece_id=AJ3Q3UP6ML4UPYVLM3VPRMNXULF4P5I57UP5RWBWA66QYKAESHEA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.066538182 (60h56m6.528630665s) [noise=true , address=89.149.201.147:4019 , piece_id=HYDF7GUBYX2ZRSSPNCRKIFDST4FB6WOHEAPYHPMXUYXI2OKCZ5EA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.068616490 (60h56m6.530708963s) [noise=true , address=81.171.14.70:4033 , piece_id=MVFXRRZN43QDVZKTVD3NHO77FDISNOALQUY4UMKMFZVNL67DAIMA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.075913866 (60h56m6.538006349s) [noise=true , address=81.171.14.70:4042 , piece_id=QCOZHOPJT52L5JHTCVPMYISO2UJMBZXXKZXMNPCX4XSJZBGGIF2A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.076377048 (60h56m6.538469531s) [noise=true , address=89.149.201.147:4009 , piece_id=F4MNYJK2MEMWWDKXCZHKVMJSYCC7CVRCHNF4TP2YEOYRQ3K7HFQA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.084005253 (60h56m6.546097747s) [noise=true , address=81.171.14.70:4027 , piece_id=TGOV2RGCAPVKOKDEPXHOWCJ4QFVMHSINAZ2DYUVE5M656SAHIHOA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.094698566 (60h56m6.556791049s) [noise=true , address=89.149.201.147:4021 , piece_id=BDPAYURJ6Z7MGVPK3CR76HZDJ32XZWQHO4WRX63G3GVNUVRMTPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.123121630 (60h56m6.585214103s) [noise=true , address=89.149.201.147:4003 , piece_id=NCSA6HS2G7MLPGYB4AV6IFPQ4ZIJ4UXYPMOL34K47MOH2QHGOVHA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.139178076 (60h56m6.601270559s) [noise=true , address=89.149.201.147:4005 , piece_id=MUOSQMC2MT4VMHRNWYVEEKSETX77CPJJWPXPSUAMXCQJ6XNGWETA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.145390767 (60h56m6.60748325s) [noise=true , address=89.149.201.147:4001 , piece_id=3JQUPRAOJWKUE4YFNW6JXO2TBEHDDESRNS722NFI2P32KEP7FXEQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.153456395 (60h56m6.615548878s) [noise=true , address=89.149.201.147:4023 , piece_id=YQ5PBWVNU3O5K4BSWOR2L63PWP5MON4L3RKVM3NGZAYO2SYHGJYQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.309249281 (60h56m6.771341755s) [noise=true , address=89.149.201.147:4013 , piece_id=OQ5RI7MLAC4A7STBD2IRCNKHUJT3CCKHXTEC34IMZUHAH2TJRPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.557152347 (60h56m7.01924483s) [noise=true , address=81.171.14.70:4043 , piece_id=M4QO4GG7R5PIIK6XG64UNYA6KYWVCBJLIE4Y2YVIE7ORL4BWTXRQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:28:19.148586872 (60h57m4.610679355s) [noise=true , address=185.167.204.205:4016 , piece_id=BBVFJ5TUAQY3HGVKXMAFZTSAVLSHIJD2PKXGGEWTM4RAYWAXFKJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148765031 (60h57m4.610857514s) [noise=true , address=185.167.204.206:4055 , piece_id=IQ62FJCAM7WOQPF7YHG3RQVQK6ZMBHSW6QZXPY3LVODJXSOF6GWQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148791893 (60h57m4.610884366s) [noise=true , address=185.167.204.205:4030 , piece_id=CTD4GIPS6R4VYKV3GTHIGFTTNDMBMHKSB7OYENJ6BSYVBBSOR77A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: rpc: dial tcp 185.167.204.205:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148927972 (60h57m4.611020465s) [noise=true , address=185.167.204.206:4065 , piece_id=AEO6P7V724TTNBNP2D5LRKSXQSZJDGFOXF7MQKDGJXDE3ESK32RQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oPo26KZGZr7o7M18KZiX5PMAAMwHK8FPT59y1hSLhkB6z3Lcj, address:185.167.204.206:4065): protocol: rpc: dial tcp 185.167.204.206:4065: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148936478 (60h57m4.611028961s) [noise=true , address=185.167.204.205:4007 , piece_id=43YBEPUGDIG6G2N2X6WORSU3U442H63OQZK34L7CVS6CK7ZS3FIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: rpc: dial tcp 185.167.204.205:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149093386 (60h57m4.611185869s) [noise=true , address=185.167.204.206:4056 , piece_id=CGFZ3N2VWSSINLTQGUXQTKD3F2RCHBVAAIJ6UHKHLMWDGFTN4SIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, address:185.167.204.206:4056): protocol: rpc: dial tcp 185.167.204.206:4056: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149109788 (60h57m4.611202281s) [noise=true , address=185.167.204.205:4010 , piece_id=7HM6R5VGM6I5CBYDWEB5QAFTMVQODXAGNGO44XKPWWC6V2BF4LZA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: rpc: dial tcp 185.167.204.205:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149259012 (60h57m4.611351505s) [noise=true , address=185.167.204.206:4059 , piece_id=FU7GCK4MIZLDAKCCPXM4QBPE5IHQILHOOJ4XI2T7ZBL3V35NCCMQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149416773 (60h57m4.611509256s) [noise=true , address=185.167.204.206:4062 , piece_id=GCY6UTKPG75JLO36FPZDMFTZWXKL5KEMU2RNB6JBU6VBX7DN3NBQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: rpc: dial tcp 185.167.204.206:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149468791 (60h57m4.611561264s) [noise=true , address=185.167.204.206:4068 , piece_id=MSUITRFBTAFTAOF7SPTQQ7FL3CTFPDGAQG4EEHMXELSCHFWT6WJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149493078 (60h57m4.611585561s) [noise=true , address=185.167.204.206:4043 , piece_id=XTDOMR7BANJPW4PCBMFGLNXRNRQCZYXAWRWOLPLNSF3KK7ONLCKQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: rpc: dial tcp 185.167.204.206:4043: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149500752 (60h57m4.611593235s) [noise=true , address=185.167.204.205:4018 , piece_id=F5OONAIPFWEWJLIZ4UTY5T6526TWHRGWRMSZPHHBYOHCOU6NYPQQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149619167 (60h57m4.61171164s) [noise=true , address=185.167.204.205:4013 , piece_id=6JMD63X4E72LNIZZ55MZ47WWWN2SWRRQNP75C3PZI2CPPVYMLAGA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 188.166.225.170:52378->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149677208 (60h57m4.611769701s) [noise=true , address=185.167.204.206:4054 , piece_id=KLRRYM3F564DQ3BILX33ZBAQNS4OWCIDXKQ7YCBPBGGADJ3R5JLA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u, address:185.167.204.206:4054): protocol: write tcp 188.166.225.170:32858->185.167.204.206:4054: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149731772 (60h57m4.611824245s) [noise=true , address=185.167.204.206:4071 , piece_id=FZ666R6LTM2B2ZSDLLP7XB3NDHMCVABOKVRMUXOHHERM2JU7FU7Q, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149757931 (60h57m4.611850404s) [seg_pos={0 0}, upload=53621]: Done waiting for segment. successful: 13 optimal: 26 errs: <nil> Sep 26 02:28:19.318741697 (60h57m4.78083418s) [upload=53621]: Done uploading segments... Sep 26 02:28:19.318776353 (60h57m4.780868836s) [upload=53621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 13) Duration: 4.849416529s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 4.947889963s FAIL (2025-09-26 02:28:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 13) ======LOGS====== gment Sep 26 02:27:20.647600085 (60h56m6.109692568s) [upload=53621]: Waiting for error group managing segments... Sep 26 02:27:21.064033182 (60h56m6.526125665s) [noise=true , address=81.171.14.70:4038 , piece_id=AJ3Q3UP6ML4UPYVLM3VPRMNXULF4P5I57UP5RWBWA66QYKAESHEA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.066538182 (60h56m6.528630665s) [noise=true , address=89.149.201.147:4019 , piece_id=HYDF7GUBYX2ZRSSPNCRKIFDST4FB6WOHEAPYHPMXUYXI2OKCZ5EA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.068616490 (60h56m6.530708963s) [noise=true , address=81.171.14.70:4033 , piece_id=MVFXRRZN43QDVZKTVD3NHO77FDISNOALQUY4UMKMFZVNL67DAIMA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.075913866 (60h56m6.538006349s) [noise=true , address=81.171.14.70:4042 , piece_id=QCOZHOPJT52L5JHTCVPMYISO2UJMBZXXKZXMNPCX4XSJZBGGIF2A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.076377048 (60h56m6.538469531s) [noise=true , address=89.149.201.147:4009 , piece_id=F4MNYJK2MEMWWDKXCZHKVMJSYCC7CVRCHNF4TP2YEOYRQ3K7HFQA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.084005253 (60h56m6.546097747s) [noise=true , address=81.171.14.70:4027 , piece_id=TGOV2RGCAPVKOKDEPXHOWCJ4QFVMHSINAZ2DYUVE5M656SAHIHOA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.094698566 (60h56m6.556791049s) [noise=true , address=89.149.201.147:4021 , piece_id=BDPAYURJ6Z7MGVPK3CR76HZDJ32XZWQHO4WRX63G3GVNUVRMTPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.123121630 (60h56m6.585214103s) [noise=true , address=89.149.201.147:4003 , piece_id=NCSA6HS2G7MLPGYB4AV6IFPQ4ZIJ4UXYPMOL34K47MOH2QHGOVHA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.139178076 (60h56m6.601270559s) [noise=true , address=89.149.201.147:4005 , piece_id=MUOSQMC2MT4VMHRNWYVEEKSETX77CPJJWPXPSUAMXCQJ6XNGWETA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.145390767 (60h56m6.60748325s) [noise=true , address=89.149.201.147:4001 , piece_id=3JQUPRAOJWKUE4YFNW6JXO2TBEHDDESRNS722NFI2P32KEP7FXEQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.153456395 (60h56m6.615548878s) [noise=true , address=89.149.201.147:4023 , piece_id=YQ5PBWVNU3O5K4BSWOR2L63PWP5MON4L3RKVM3NGZAYO2SYHGJYQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.309249281 (60h56m6.771341755s) [noise=true , address=89.149.201.147:4013 , piece_id=OQ5RI7MLAC4A7STBD2IRCNKHUJT3CCKHXTEC34IMZUHAH2TJRPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.557152347 (60h56m7.01924483s) [noise=true , address=81.171.14.70:4043 , piece_id=M4QO4GG7R5PIIK6XG64UNYA6KYWVCBJLIE4Y2YVIE7ORL4BWTXRQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:28:19.148586872 (60h57m4.610679355s) [noise=true , address=185.167.204.205:4016 , piece_id=BBVFJ5TUAQY3HGVKXMAFZTSAVLSHIJD2PKXGGEWTM4RAYWAXFKJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148765031 (60h57m4.610857514s) [noise=true , address=185.167.204.206:4055 , piece_id=IQ62FJCAM7WOQPF7YHG3RQVQK6ZMBHSW6QZXPY3LVODJXSOF6GWQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148791893 (60h57m4.610884366s) [noise=true , address=185.167.204.205:4030 , piece_id=CTD4GIPS6R4VYKV3GTHIGFTTNDMBMHKSB7OYENJ6BSYVBBSOR77A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: rpc: dial tcp 185.167.204.205:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148927972 (60h57m4.611020465s) [noise=true , address=185.167.204.206:4065 , piece_id=AEO6P7V724TTNBNP2D5LRKSXQSZJDGFOXF7MQKDGJXDE3ESK32RQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oPo26KZGZr7o7M18KZiX5PMAAMwHK8FPT59y1hSLhkB6z3Lcj, address:185.167.204.206:4065): protocol: rpc: dial tcp 185.167.204.206:4065: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148936478 (60h57m4.611028961s) [noise=true , address=185.167.204.205:4007 , piece_id=43YBEPUGDIG6G2N2X6WORSU3U442H63OQZK34L7CVS6CK7ZS3FIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: rpc: dial tcp 185.167.204.205:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149093386 (60h57m4.611185869s) [noise=true , address=185.167.204.206:4056 , piece_id=CGFZ3N2VWSSINLTQGUXQTKD3F2RCHBVAAIJ6UHKHLMWDGFTN4SIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, address:185.167.204.206:4056): protocol: rpc: dial tcp 185.167.204.206:4056: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149109788 (60h57m4.611202281s) [noise=true , address=185.167.204.205:4010 , piece_id=7HM6R5VGM6I5CBYDWEB5QAFTMVQODXAGNGO44XKPWWC6V2BF4LZA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: rpc: dial tcp 185.167.204.205:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149259012 (60h57m4.611351505s) [noise=true , address=185.167.204.206:4059 , piece_id=FU7GCK4MIZLDAKCCPXM4QBPE5IHQILHOOJ4XI2T7ZBL3V35NCCMQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149416773 (60h57m4.611509256s) [noise=true , address=185.167.204.206:4062 , piece_id=GCY6UTKPG75JLO36FPZDMFTZWXKL5KEMU2RNB6JBU6VBX7DN3NBQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: rpc: dial tcp 185.167.204.206:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149468791 (60h57m4.611561264s) [noise=true , address=185.167.204.206:4068 , piece_id=MSUITRFBTAFTAOF7SPTQQ7FL3CTFPDGAQG4EEHMXELSCHFWT6WJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149493078 (60h57m4.611585561s) [noise=true , address=185.167.204.206:4043 , piece_id=XTDOMR7BANJPW4PCBMFGLNXRNRQCZYXAWRWOLPLNSF3KK7ONLCKQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: rpc: dial tcp 185.167.204.206:4043: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149500752 (60h57m4.611593235s) [noise=true , address=185.167.204.205:4018 , piece_id=F5OONAIPFWEWJLIZ4UTY5T6526TWHRGWRMSZPHHBYOHCOU6NYPQQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149619167 (60h57m4.61171164s) [noise=true , address=185.167.204.205:4013 , piece_id=6JMD63X4E72LNIZZ55MZ47WWWN2SWRRQNP75C3PZI2CPPVYMLAGA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 188.166.225.170:52378->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149677208 (60h57m4.611769701s) [noise=true , address=185.167.204.206:4054 , piece_id=KLRRYM3F564DQ3BILX33ZBAQNS4OWCIDXKQ7YCBPBGGADJ3R5JLA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u, address:185.167.204.206:4054): protocol: write tcp 188.166.225.170:32858->185.167.204.206:4054: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149731772 (60h57m4.611824245s) [noise=true , address=185.167.204.206:4071 , piece_id=FZ666R6LTM2B2ZSDLLP7XB3NDHMCVABOKVRMUXOHHERM2JU7FU7Q, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149757931 (60h57m4.611850404s) [seg_pos={0 0}, upload=53621]: Done waiting for segment. successful: 13 optimal: 26 errs: <nil> Sep 26 02:28:19.318741697 (60h57m4.78083418s) [upload=53621]: Done uploading segments... Sep 26 02:28:19.318776353 (60h57m4.780868836s) [upload=53621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 13) Duration: 4.947913017s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.171741299s FAIL (2025-09-26 02:28:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 13) ======LOGS====== gment Sep 26 02:27:20.647600085 (60h56m6.109692568s) [upload=53621]: Waiting for error group managing segments... Sep 26 02:27:21.064033182 (60h56m6.526125665s) [noise=true , address=81.171.14.70:4038 , piece_id=AJ3Q3UP6ML4UPYVLM3VPRMNXULF4P5I57UP5RWBWA66QYKAESHEA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.066538182 (60h56m6.528630665s) [noise=true , address=89.149.201.147:4019 , piece_id=HYDF7GUBYX2ZRSSPNCRKIFDST4FB6WOHEAPYHPMXUYXI2OKCZ5EA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.068616490 (60h56m6.530708963s) [noise=true , address=81.171.14.70:4033 , piece_id=MVFXRRZN43QDVZKTVD3NHO77FDISNOALQUY4UMKMFZVNL67DAIMA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.075913866 (60h56m6.538006349s) [noise=true , address=81.171.14.70:4042 , piece_id=QCOZHOPJT52L5JHTCVPMYISO2UJMBZXXKZXMNPCX4XSJZBGGIF2A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.076377048 (60h56m6.538469531s) [noise=true , address=89.149.201.147:4009 , piece_id=F4MNYJK2MEMWWDKXCZHKVMJSYCC7CVRCHNF4TP2YEOYRQ3K7HFQA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.084005253 (60h56m6.546097747s) [noise=true , address=81.171.14.70:4027 , piece_id=TGOV2RGCAPVKOKDEPXHOWCJ4QFVMHSINAZ2DYUVE5M656SAHIHOA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.094698566 (60h56m6.556791049s) [noise=true , address=89.149.201.147:4021 , piece_id=BDPAYURJ6Z7MGVPK3CR76HZDJ32XZWQHO4WRX63G3GVNUVRMTPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.123121630 (60h56m6.585214103s) [noise=true , address=89.149.201.147:4003 , piece_id=NCSA6HS2G7MLPGYB4AV6IFPQ4ZIJ4UXYPMOL34K47MOH2QHGOVHA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.139178076 (60h56m6.601270559s) [noise=true , address=89.149.201.147:4005 , piece_id=MUOSQMC2MT4VMHRNWYVEEKSETX77CPJJWPXPSUAMXCQJ6XNGWETA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.145390767 (60h56m6.60748325s) [noise=true , address=89.149.201.147:4001 , piece_id=3JQUPRAOJWKUE4YFNW6JXO2TBEHDDESRNS722NFI2P32KEP7FXEQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.153456395 (60h56m6.615548878s) [noise=true , address=89.149.201.147:4023 , piece_id=YQ5PBWVNU3O5K4BSWOR2L63PWP5MON4L3RKVM3NGZAYO2SYHGJYQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.309249281 (60h56m6.771341755s) [noise=true , address=89.149.201.147:4013 , piece_id=OQ5RI7MLAC4A7STBD2IRCNKHUJT3CCKHXTEC34IMZUHAH2TJRPIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:27:21.557152347 (60h56m7.01924483s) [noise=true , address=81.171.14.70:4043 , piece_id=M4QO4GG7R5PIIK6XG64UNYA6KYWVCBJLIE4Y2YVIE7ORL4BWTXRQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: <nil> Sep 26 02:28:19.148586872 (60h57m4.610679355s) [noise=true , address=185.167.204.205:4016 , piece_id=BBVFJ5TUAQY3HGVKXMAFZTSAVLSHIJD2PKXGGEWTM4RAYWAXFKJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148765031 (60h57m4.610857514s) [noise=true , address=185.167.204.206:4055 , piece_id=IQ62FJCAM7WOQPF7YHG3RQVQK6ZMBHSW6QZXPY3LVODJXSOF6GWQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148791893 (60h57m4.610884366s) [noise=true , address=185.167.204.205:4030 , piece_id=CTD4GIPS6R4VYKV3GTHIGFTTNDMBMHKSB7OYENJ6BSYVBBSOR77A, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: rpc: dial tcp 185.167.204.205:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148927972 (60h57m4.611020465s) [noise=true , address=185.167.204.206:4065 , piece_id=AEO6P7V724TTNBNP2D5LRKSXQSZJDGFOXF7MQKDGJXDE3ESK32RQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oPo26KZGZr7o7M18KZiX5PMAAMwHK8FPT59y1hSLhkB6z3Lcj, address:185.167.204.206:4065): protocol: rpc: dial tcp 185.167.204.206:4065: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.148936478 (60h57m4.611028961s) [noise=true , address=185.167.204.205:4007 , piece_id=43YBEPUGDIG6G2N2X6WORSU3U442H63OQZK34L7CVS6CK7ZS3FIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: rpc: dial tcp 185.167.204.205:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149093386 (60h57m4.611185869s) [noise=true , address=185.167.204.206:4056 , piece_id=CGFZ3N2VWSSINLTQGUXQTKD3F2RCHBVAAIJ6UHKHLMWDGFTN4SIQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, address:185.167.204.206:4056): protocol: rpc: dial tcp 185.167.204.206:4056: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149109788 (60h57m4.611202281s) [noise=true , address=185.167.204.205:4010 , piece_id=7HM6R5VGM6I5CBYDWEB5QAFTMVQODXAGNGO44XKPWWC6V2BF4LZA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: rpc: dial tcp 185.167.204.205:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149259012 (60h57m4.611351505s) [noise=true , address=185.167.204.206:4059 , piece_id=FU7GCK4MIZLDAKCCPXM4QBPE5IHQILHOOJ4XI2T7ZBL3V35NCCMQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149416773 (60h57m4.611509256s) [noise=true , address=185.167.204.206:4062 , piece_id=GCY6UTKPG75JLO36FPZDMFTZWXKL5KEMU2RNB6JBU6VBX7DN3NBQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: rpc: dial tcp 185.167.204.206:4062: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149468791 (60h57m4.611561264s) [noise=true , address=185.167.204.206:4068 , piece_id=MSUITRFBTAFTAOF7SPTQQ7FL3CTFPDGAQG4EEHMXELSCHFWT6WJA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149493078 (60h57m4.611585561s) [noise=true , address=185.167.204.206:4043 , piece_id=XTDOMR7BANJPW4PCBMFGLNXRNRQCZYXAWRWOLPLNSF3KK7ONLCKQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: rpc: dial tcp 185.167.204.206:4043: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149500752 (60h57m4.611593235s) [noise=true , address=185.167.204.205:4018 , piece_id=F5OONAIPFWEWJLIZ4UTY5T6526TWHRGWRMSZPHHBYOHCOU6NYPQQ, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149619167 (60h57m4.61171164s) [noise=true , address=185.167.204.205:4013 , piece_id=6JMD63X4E72LNIZZ55MZ47WWWN2SWRRQNP75C3PZI2CPPVYMLAGA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 188.166.225.170:52378->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149677208 (60h57m4.611769701s) [noise=true , address=185.167.204.206:4054 , piece_id=KLRRYM3F564DQ3BILX33ZBAQNS4OWCIDXKQ7YCBPBGGADJ3R5JLA, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u, address:185.167.204.206:4054): protocol: write tcp 188.166.225.170:32858->185.167.204.206:4054: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 26 02:28:19.149731772 (60h57m4.611824245s) [noise=true , address=185.167.204.206:4071 , piece_id=FZ666R6LTM2B2ZSDLLP7XB3NDHMCVABOKVRMUXOHHERM2JU7FU7Q, seg_pos={0 0}, upload=53621]: Done uploading piece. err: ecclient: upload failed (node:1D11dtZNczougJ1KxaPqpiKAREiUZW6P7UenqMkLZzqDEEvjK8, address:185.167.204.206:4071): protocol: context deadline exceeded; context deadline exceeded Sep 26 02:28:19.149757931 (60h57m4.611850404s) [seg_pos={0 0}, upload=53621]: Done waiting for segment. successful: 13 optimal: 26 errs: <nil> Sep 26 02:28:19.318741697 (60h57m4.78083418s) [upload=53621]: Done uploading segments... Sep 26 02:28:19.318776353 (60h57m4.780868836s) [upload=53621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 13) Duration: 1m0.171767821s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.858601551s FAIL (2025-09-25 23:17:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.858629474s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.262704553s FAIL (2025-09-25 23:16:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.262757694s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.886452244s FAIL (2025-09-25 23:16:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.886479346s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.019162681s FAIL (2025-09-25 23:15:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.019198729s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.068122641s FAIL (2025-09-25 23:13:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.06821624s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.036584724s FAIL (2025-09-25 23:13:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.036634068s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.133974581s FAIL (2025-09-25 23:12:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.134031589s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.012646124s FAIL (2025-09-25 23:11:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.012675771s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.004252344s FAIL (2025-09-25 23:11:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.004283983s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.947055821s FAIL (2025-09-25 23:11:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.947080468s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.88393615s FAIL (2025-09-25 23:10:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.883965376s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.072474249s FAIL (2025-09-25 23:09:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.07250047s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 183.91529ms remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Content-Length=352 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=JdrtmpHLrtw X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1868A6DD9BAF33BB Date=Thu, 25 Sep 2025 22:25:25 GMT target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-09-25 23:09:00 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: context canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1868A6DD9BAF33BB</RequestId><HostId></HostId></Error>" Duration: 183.94663ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.959553951s FAIL (2025-09-25 23:08:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.959582295s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 234.866769ms remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Content-Length=352 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=JdrtmpHLrtw X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1868A6DD9BAF33BB Date=Thu, 25 Sep 2025 22:25:25 GMT target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-09-25 23:08:41 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: context canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1868A6DD9BAF33BB</RequestId><HostId></HostId></Error>" Duration: 234.90427ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.004682621s FAIL (2025-09-25 23:08:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.004719209s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 174.55195ms remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Content-Length=352 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=JdrtmpHLrtw X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1868A6DD9BAF33BB Date=Thu, 25 Sep 2025 22:25:25 GMT target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-09-25 23:07:52 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: context canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1868A6DD9BAF33BB</RequestId><HostId></HostId></Error>" Duration: 174.579502ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.191484803s FAIL (2025-09-25 23:07:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.191518808s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 177.977714ms remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Content-Length=352 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=JdrtmpHLrtw X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1868A6DD9BAF33BB Date=Thu, 25 Sep 2025 22:25:25 GMT target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-09-25 23:07:16 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: context canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1868A6DD9BAF33BB</RequestId><HostId></HostId></Error>" Duration: 178.00748ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 203.396696ms remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Content-Length=352 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=JdrtmpHLrtw X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1868A6DD9BAF33BB Date=Thu, 25 Sep 2025 22:25:25 GMT target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-09-25 23:06:57 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: context canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1868A6DD9BAF33BB</RequestId><HostId></HostId></Error>" Duration: 203.423637ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.08020839s FAIL (2025-09-25 23:06:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.080269336s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 182.713552ms remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Content-Length=352 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=JdrtmpHLrtw X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1868A6DD9BAF33BB Date=Thu, 25 Sep 2025 22:25:25 GMT target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-09-25 23:06:23 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: context canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1868A6DD9BAF33BB</RequestId><HostId></HostId></Error>" Duration: 182.744993ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.972345464s FAIL (2025-09-25 23:06:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.972372536s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 228.517157ms remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Content-Length=352 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=JdrtmpHLrtw X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1868A6DD9BAF33BB Date=Thu, 25 Sep 2025 22:25:25 GMT target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-09-25 23:05:44 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: context canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1868A6DD9BAF33BB</RequestId><HostId></HostId></Error>" Duration: 228.545281ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.088753857s FAIL (2025-09-25 23:05:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.088782803s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 174.536201ms remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Content-Length=352 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=JdrtmpHLrtw X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1868A6DD9BAF33BB Date=Thu, 25 Sep 2025 22:25:25 GMT target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-09-25 23:04:59 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: context canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1868A6DD9BAF33BB</RequestId><HostId></HostId></Error>" Duration: 174.568873ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.106803881s FAIL (2025-09-25 23:04:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.106835972s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.923853279s FAIL (2025-09-25 23:04:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.923879179s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 174.211442ms remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Content-Length=352 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=JdrtmpHLrtw X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1868A6DD9BAF33BB Date=Thu, 25 Sep 2025 22:25:25 GMT target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-09-25 23:04:10 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: context canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1868A6DD9BAF33BB</RequestId><HostId></HostId></Error>" Duration: 174.238294ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.073078292s FAIL (2025-09-25 23:03:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.073106116s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 183.517603ms remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Content-Length=352 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=JdrtmpHLrtw X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1868A6DD9BAF33BB Date=Thu, 25 Sep 2025 22:25:25 GMT target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-09-25 23:03:21 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: context canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1868A6DD9BAF33BB</RequestId><HostId></HostId></Error>" Duration: 183.565764ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.971784995s FAIL (2025-09-25 23:03:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.971815513s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 220.253573ms remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Content-Length=352 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=JdrtmpHLrtw X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1868A6DD9BAF33BB Date=Thu, 25 Sep 2025 22:25:25 GMT target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-09-25 23:02:30 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: context canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1868A6DD9BAF33BB</RequestId><HostId></HostId></Error>" Duration: 220.281938ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.142359827s FAIL (2025-09-25 23:02:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.142391937s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.070820092s FAIL (2025-09-25 23:01:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.070852675s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 179.968385ms remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Content-Length=352 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=JdrtmpHLrtw X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1868A6DD9BAF33BB Date=Thu, 25 Sep 2025 22:25:25 GMT target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-09-25 23:01:29 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: context canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1868A6DD9BAF33BB</RequestId><HostId></HostId></Error>" Duration: 179.992741ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.127621331s FAIL (2025-09-25 23:01:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.127658864s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 187.009352ms remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Content-Length=352 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=JdrtmpHLrtw X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1868A6DD9BAF33BB Date=Thu, 25 Sep 2025 22:25:25 GMT target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 FAIL (2025-09-25 23:00:53 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: context canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1868A6DD9BAF33BB</RequestId><HostId></HostId></Error>" Duration: 187.03505ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.082565236s FAIL (2025-09-25 23:00:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 25 22:33:20.609515379 (57h2m6.071607862s) [upload=50198]: Starting upload Sep 25 22:33:20.609634665 (57h2m6.071727138s) [upload=50198]: Done starting upload Sep 25 22:33:20.609747009 (57h2m6.071839492s) [upload=50198]: Uploading segments... Sep 25 22:33:20.609932132 (57h2m6.072024605s) [upload=50198]: Got next segment. Inline: false Sep 25 22:33:20.609977649 (57h2m6.072070132s) [upload=50198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 25 22:33:20.609983990 (57h2m6.072076473s) [upload=50198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 25 22:33:20.876293406 (57h2m6.338385889s) [seg_pos={0 0}, upload=50198]: Begin upload segment... Sep 25 22:33:20.876365564 (57h2m6.338458047s) [seg_pos={0 0}, upload=50198]: Done begin upload segment. Sep 25 22:33:20.876398446 (57h2m6.338490919s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876412874 (57h2m6.338505347s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876460024 (57h2m6.338552507s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.876711823 (57h2m6.338804296s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877150188 (57h2m6.339242661s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877621355 (57h2m6.339713838s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.877942707 (57h2m6.340035181s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878252356 (57h2m6.340344829s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.878573619 (57h2m6.340666092s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.881006501 (57h2m6.343098975s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882423098 (57h2m6.344515571s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.882592721 (57h2m6.344685194s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.885196980 (57h2m6.347289464s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.888905401 (57h2m6.350997874s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.891203167 (57h2m6.35329565s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:20.892678096 (57h2m6.354770569s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Uploading piece... Sep 25 22:33:22.883631748 (57h2m8.345724231s) [upload=50198]: Next returned nil segment Sep 25 22:33:22.883655924 (57h2m8.345748397s) [upload=50198]: Waiting for error group managing segments... Sep 25 22:33:23.186669559 (57h2m8.648762032s) [noise=true , address=77.90.31.15:4697 , piece_id=RRP4PCTAVSJYH377LRZ43I36D4LWG7EPHE23LDBUZLP2DUR3RNDQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.402505165 (57h2m8.864597638s) [noise=true , address=45.91.116.7:4010 , piece_id=QZEN3HHQVP5SQ3ABXX6AUG2T7V2JTJYLKXPEM2FCIWEIRSZ2IOUA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.460847597 (57h2m8.92294007s) [noise=true , address=45.91.116.4:4939 , piece_id=7UY722GYY34MQHO6FCVNCMXKV67RUX2TTNXGUXGGDEBE63HPI6DQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:23.475828172 (57h2m8.937920645s) [noise=true , address=45.91.116.10:4082 , piece_id=OJUG2YVVDOHECUD3DLDJANOWGXKS76BD3KIWFF5Q2OTRELONDWUQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:41.316700325 (57h2m26.778792808s) [noise=true , address=145.79.227.18:4007 , piece_id=OWW73ZTC3OLJTGC4IWXZ44VT4ERGUE4YGGULZCSJKYEHNXRNLLRA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:42.134617358 (57h2m27.596709841s) [noise=true , address=145.79.227.11:4729 , piece_id=WTDDZ72XM433FAERK56RR7BWIFEFFMKJ7PKKIQQUCLWZTTTTB5TA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.316084677 (57h2m28.77817716s) [noise=true , address=145.79.227.15:4874 , piece_id=I5VJYG36ZG63ZRJUC2WKXISA5C7EJQSV3XZJ7FLN5MWMC3CDBNEA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:43.336895665 (57h2m28.798988148s) [noise=true , address=145.79.227.15:4873 , piece_id=C3JJ5HPO237IQRGLKQ5QZSKEJM2HNOH4YNBMULOE7CXLFO4UEW3Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:44.766574920 (57h2m30.228667393s) [noise=true , address=145.79.227.17:4972 , piece_id=OLLDZMGRQBTZAQURMSU4AVHOAFCD3VBWCH6Y5WQXENHW2WB3D6BQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:47.245353345 (57h2m32.707445818s) [noise=true , address=145.79.227.13:4802 , piece_id=UNCMDLJ7BWSS2SOAXZRUYR7EQTKZWOYFHJSVB22DHNH32VADTPNA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:48.389979384 (57h2m33.852071867s) [noise=true , address=145.79.227.19:4079 , piece_id=TSXXBPAF2KFTUP2MDC6NKGLP2PF666PMSXPGYYKDPO25FRF7YQ6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.870866176 (57h2m35.332958649s) [noise=true , address=145.79.227.3:4151 , piece_id=ZMJ7WBX4DWOWTTJTLJXFZKBR4MRE2RS2FQWRM2YZUC6NHVEAUZFA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: <nil> Sep 25 22:33:49.927808223 (57h2m35.389900706s) [noise=true , address=145.79.227.2:4115 , piece_id=GDDMOKVGUSPCUNXCYSADTSDEWV3WX6YLR3HSMC72AI7345WMWI6Q, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927871833 (57h2m35.389964316s) [noise=true , address=145.79.227.6:4259 , piece_id=VFXMPOPGL2UF6D6NLM7PW3GKM5YEQHT3ZDMZGOR5VEX3NNLOAXAQ, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927907451 (57h2m35.389999924s) [noise=true , address=145.79.227.4:4185 , piece_id=UDAN2TY4RX3ATTJAYBL5G3ZXKLYIXFIBPYJWRIG2MPD3LJH62UGA, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46508->145.79.227.4:4185: use of closed network connection Sep 25 22:33:49.927937458 (57h2m35.390029931s) [noise=true , address=145.79.227.6:4257 , piece_id=NLAZ3I5A6RU3PWBBSMICAETTWZ7UZLO6X5VEXUZPZ2ZB4KAI6I5A, seg_pos={0 0}, upload=50198]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Sep 25 22:33:49.927971413 (57h2m35.390063886s) [seg_pos={0 0}, upload=50198]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 25 22:33:50.167900726 (57h2m35.629993199s) [upload=50198]: Done uploading segments... Sep 25 22:33:50.167951733 (57h2m35.630044216s) [upload=50198]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.082662902s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)