Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.31450496s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:03:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 6.314542141s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.372489404s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:02:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.372518221s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.820808592s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:02:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.820843829s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.807785408s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:01:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.80781204s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.027239415s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:00:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 6.027276536s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.160719404s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:59:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 6.160826568s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.201669968s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:58:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 6.201702699s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.817240065s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:57:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.81726392s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.325359547s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:56:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.325392892s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.718760399s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:56:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 6.718796346s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.726074864s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:55:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.726105442s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.667205574s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:54:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.66723552s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.734096672s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:54:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.734188365s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.247516642s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:53:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 6.247546508s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.820385686s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:52:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.820414971s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.864555011s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:52:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.864586732s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.285874069s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:52:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.285903807s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.764991487s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:51:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.765022418s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.354222691s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:51:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.354251966s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.326250132s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:50:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.326281722s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.388848129s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:50:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.389305481s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.663083034s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:49:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.663109495s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.815854416s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:48:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.815884695s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.759331812s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:47:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 6.759366639s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.775764887s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:46:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.775790816s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.86304743s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:45:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.863074062s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 13.609931634s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:45:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 13.609964308s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.829292434s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:44:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.829310449s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.17640712s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 07:44:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 1m0.176435383s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.878343528s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:19:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.878380398s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.776489191s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:18:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.776523104s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.02215943s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:17:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.022190509s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.957652055s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:16:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.957683155s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.002142884s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:15:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.002170357s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.063512383s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:15:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.06375207s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.748347603s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:14:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.748381378s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.941349863s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:14:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.941381204s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.044850627s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:13:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.044971566s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.033114868s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:13:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.033150065s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.036115707s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:12:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.036144693s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.010304277s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:11:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.010329657s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.959143659s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:11:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.959174499s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.857664036s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:11:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.857947376s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.933083868s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:10:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.933211862s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.000632853s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:09:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.000748894s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.047379837s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:08:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.047408092s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.83144613s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:07:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.831473985s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.975695207s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:07:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.975725926s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.91183844s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:06:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.911870862s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.126082871s trace: https://jaeger.dev.storj.io/trace/7f7f7ac45d485d24 FAIL (2025-11-21 06:05:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792245668 (285h29m51.343716129s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.792885526 (285h29m51.344355987s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793151482 (285h29m51.344621943s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793491901 (285h29m51.344962362s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.793891742 (285h29m51.345362193s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794141177 (285h29m51.345611638s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794307884 (285h29m51.345778345s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794735548 (285h29m51.346206009s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.794907846 (285h29m51.346378307s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795164165 (285h29m51.346634626s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795628088 (285h29m51.347098559s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.795709643 (285h29m51.347180104s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796259280 (285h29m51.347729731s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.796837141 (285h29m51.348307602s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:52.797554957 (285h29m51.349025418s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Uploading piece... Nov 21 05:33:54.639061227 (285h29m53.190531678s) [upload=251442]: Next returned nil segment Nov 21 05:33:54.639076927 (285h29m53.190547378s) [upload=251442]: Waiting for error group managing segments... Nov 21 05:33:54.947232317 (285h29m53.498702768s) [noise=true , address=77.90.31.21:4879 , piece_id=Q6SWAMZ6LWOAY5KO2FZXVIY5PRPOXKNB6REVD64JW3BHYW7EGK5A, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.965825712 (285h29m53.517296163s) [noise=true , address=77.90.31.19:4843 , piece_id=3NZRDZTO47C6CKIGJFISD5UNS5CCLNIXNLFCEM3SHMALJD445IUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:54.970024989 (285h29m53.52149544s) [noise=true , address=45.91.116.10:4082 , piece_id=4XKYOVK7GBERU5N6ZBGDSNX2FIGPWYWESGZAPJB5OTFX43BGMBJA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:33:59.152863892 (285h29m57.704334353s) [noise=true , address=77.90.31.17:4770 , piece_id=EBHOM26LVHLK623EFTYAIUYSDFC2FXPPDM6EBANWU6VKHVIV35XA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:09.387807677 (285h30m7.939278138s) [noise=true , address=145.79.227.31:4798 , piece_id=E5ZNENRHLALVZ24H3JRGPDQG2QTRWF5W4XZGTIUWEH54EZHGF5FQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:16.338520492 (285h30m14.889990953s) [noise=true , address=145.79.227.20:4439 , piece_id=3GF4ONY37VYE47G6RXD2ZHGUAX5G4Y2JJB4NFJXQMZOLIWHGWMTQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.121115252 (285h30m19.672585723s) [noise=true , address=145.79.227.23:4546 , piece_id=57V6BYGRJ3K5KSPYJSY3ETC4SVCWWCX4RSW2RATL5K3H3F2M6XWQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.316588968 (285h30m19.868059429s) [noise=true , address=145.79.227.10:4402 , piece_id=64IKYGGISAHJ6DA2TIBQYC3WSR4WEAY6XXUZFY3TVWG2ZONZYMUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: <nil> Nov 21 05:34:21.857923180 (285h30m20.409393641s) [noise=true , address=145.79.227.15:4874 , piece_id=WGF5C75CKSFQXPZHA7WZYS2E4MXWHYYH4EJX4RX7YWWNV2V2DTUA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49760->145.79.227.15:4874: use of closed network connection Nov 21 05:34:21.858107290 (285h30m20.409577751s) [noise=true , address=145.79.227.6:4257 , piece_id=SUD3NVBH57B7VB4XMA7BT5JDUIMTBNYHQ57SKS43OYPZHOSRTAOQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: expected piece hash; context deadline exceeded; EOF Nov 21 05:34:21.858285860 (285h30m20.409756321s) [noise=true , address=145.79.227.28:4725 , piece_id=ROJKSPWBWVI7XGULBFHH5QZ67VWXOIJ4CNDMONH2TTDFCY73EODQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12YKLTah3swX14kL82eNogJxd8D8Hjfq7SPjDDQAapDnXmDctP1, address:145.79.227.28:4725): protocol: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44958->145.79.227.28:4725: use of closed network connection Nov 21 05:34:21.858327278 (285h30m20.40979774s) [noise=true , address=145.79.227.27:4689 , piece_id=A4PDPACFKIFXRRWER3HTAZWSEA3XPQ3O5JGKQ6POTXD4LB5OJRDA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49336->145.79.227.27:4689: use of closed network connection Nov 21 05:34:21.858408854 (285h30m20.409879316s) [noise=true , address=145.79.227.11:4731 , piece_id=UMXIUIH44P56A4XQN5SLN3WN4H6ULV6VQEBXD2BTMJ52IC5WEN2Q, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43432->145.79.227.11:4731: use of closed network connection Nov 21 05:34:21.858603585 (285h30m20.410074046s) [noise=true , address=145.79.227.9:4366 , piece_id=OTSW62BWEAT5UI7CCFTZFSMQSMY65553YNA5JHHGX3TSIL7KZMKQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:53824->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858659682 (285h30m20.410130193s) [noise=true , address=145.79.227.14:4838 , piece_id=LHUOTXHJ6AUHEHZR74ZVARJQYKFA2S7HHKRFFXZPFL44URSMUWOA, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57202->145.79.227.14:4838: use of closed network connection Nov 21 05:34:21.858719015 (285h30m20.410189466s) [noise=true , address=145.79.227.19:4079 , piece_id=ATVDXX3GGPIOUDQ6NB7LZOBMQRGFWITTYI4RGKRLNDOZL36PYPMQ, seg_pos={0 0}, upload=251442]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:37786->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 05:34:21.858750013 (285h30m20.410220474s) [seg_pos={0 0}, upload=251442]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 21 05:34:22.100591200 (285h30m20.652061651s) [upload=251442]: Done uploading segments... Nov 21 05:34:22.100678145 (285h30m20.652148626s) [upload=251442]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.126122797s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)